2023-11-16T23:37:03.421 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-16T23:37:03.422 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_choffman@teuthology --name choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi --archive /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180 --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} -- /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/orig.config.yaml 2023-11-16T23:37:03.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T23:37:03.563 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7461180' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793-testing sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6283 sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b sleep_before_teardown: 0 suite: fs suite_branch: wip-62793-testing suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-16_21:22:00 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1224587 2023-11-16T23:37:03.563 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa; will attempt to use it 2023-11-16T23:37:03.564 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks 2023-11-16T23:37:03.564 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-16T23:37:03.566 INFO:teuthology.task.internal:Checking packages... 2023-11-16T23:37:03.603 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1c9565146c8647d4e00eb6c7ae626dc192680e7b' 2023-11-16T23:37:03.603 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-16T23:37:03.603 INFO:teuthology.packaging:ref: None 2023-11-16T23:37:03.603 INFO:teuthology.packaging:tag: None 2023-11-16T23:37:03.603 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-16T23:37:03.603 INFO:teuthology.packaging:sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:37:03.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-16T23:37:03.774 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3906.g1c956514 2023-11-16T23:37:03.774 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-16T23:37:03.780 INFO:teuthology.task.internal:no buildpackages task found 2023-11-16T23:37:03.781 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-16T23:37:03.790 INFO:teuthology.task.internal:Saving configuration 2023-11-16T23:37:03.820 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-16T23:37:03.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T23:37:03.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.707568', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T23:37:03.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.711857', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T23:37:03.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.710219', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T23:37:03.902 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-16T23:37:03.911 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-16T23:37:03.911 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-16T23:37:03.911 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-16T23:37:03.911 INFO:teuthology.run_tasks:Running task console_log... 2023-11-16T23:37:04.045 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbbcaed4f70>, signals=[15]) 2023-11-16T23:37:04.045 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-16T23:37:04.065 INFO:teuthology.task.internal:Opening connections... 2023-11-16T23:37:04.066 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2023-11-16T23:37:04.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:37:04.136 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2023-11-16T23:37:04.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:37:04.204 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-11-16T23:37:04.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:37:04.271 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-16T23:37:04.279 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-11-16T23:37:04.301 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-11-16T23:37:04.302 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-11-16T23:37:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T23:37:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T23:37:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-11-16T23:37:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-11-16T23:37:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.6" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T23:37:04.363 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T23:37:04.364 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T23:37:04.364 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-11-16T23:37:04.391 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-11-16T23:37:04.414 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-11-16T23:37:04.414 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-11-16T23:37:04.473 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T23:37:04.473 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T23:37:04.473 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T23:37:04.474 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T23:37:04.475 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T23:37:04.475 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T23:37:04.475 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-11-16T23:37:04.499 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-11-16T23:37:04.517 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-11-16T23:37:04.517 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-11-16T23:37:04.576 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T23:37:04.576 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T23:37:04.577 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-11-16T23:37:04.577 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-11-16T23:37:04.577 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-11-16T23:37:04.577 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-11-16T23:37:04.577 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T23:37:04.578 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-11-16T23:37:04.578 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T23:37:04.578 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T23:37:04.578 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T23:37:04.578 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T23:37:04.579 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:37:04.579 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T23:37:04.579 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T23:37:04.579 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T23:37:04.580 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T23:37:04.580 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-11-16T23:37:04.599 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-16T23:37:04.613 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-16T23:37:04.622 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-16T23:37:04.622 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2023-11-16T23:37:04.625 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2023-11-16T23:37:04.627 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-11-16T23:37:04.648 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-16T23:37:04.656 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-16T23:37:04.657 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2023-11-16T23:37:04.688 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2023-11-16T23:37:04.690 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-11-16T23:37:04.714 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-16T23:37:04.798 INFO:teuthology.run_tasks:Running task kernel... 2023-11-16T23:37:04.814 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-16T23:37:04.814 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-11-16T23:37:04.815 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-11-16T23:37:04.815 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T23:37:04.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:37:04.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-16T23:37:05.127 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T23:37:05.127 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-16T23:37:05.128 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-11-16T23:37:05.148 INFO:teuthology.orchestra.run.smithi006.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-16T23:37:05.148 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T23:37:05.148 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-16T23:37:05.149 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum upgrade -y linux-firmware 2023-11-16T23:37:06.373 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-16T23:37:06.373 INFO:teuthology.orchestra.run.smithi006.stdout:Unable to read consumer identity 2023-11-16T23:37:07.832 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-11-16T23:37:08.632 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-11-16T23:37:12.341 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 389 kB/s | 24 kB 00:00 2023-11-16T23:37:13.159 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-16T23:37:13.160 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-11-16T23:37:13.160 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-16T23:37:13.278 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-16T23:37:13.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-16T23:37:13.498 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ 2023-11-16T23:37:13.498 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-16T23:37:13.501 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi006.front.sepia.ceph.com to complete... 2023-11-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16 23:37:27 URL:https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [383129572/383129572] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi006.stderr:FINISHED --2023-11-16 23:37:27-- 2023-11-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi006.stderr:Total wall clock time: 14s 2023-11-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi006.stderr:Downloaded: 1 files, 365M in 14s (27.0 MB/s) 2023-11-16T23:37:27.373 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-16T23:37:27.374 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-16T23:37:29.041 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying... ######################################## 2023-11-16T23:37:29.108 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing... ######################################## 2023-11-16T23:37:29.200 INFO:teuthology.orchestra.run.smithi006.stdout:Updating / installing... 2023-11-16T23:38:10.036 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-16T23:38:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-16T23:38:35.489 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-16T23:38:35.490 DEBUG:teuthology.orchestra.run.smithi006:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-16T23:38:35.520 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-16T23:38:35.521 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-16T23:38:35.522 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-16T23:38:35.523 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-16T23:38:35.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-16T23:38:35.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-16T23:38:35.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-16T23:38:35.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-16T23:38:35.678 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-16T23:38:35.679 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-16T23:38:35.680 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-16T23:38:35.681 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-16T23:38:35.682 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-16T23:38:35.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-16T23:38:35.769 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-16T23:38:35.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-16T23:38:35.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-16T23:38:35.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-16T23:38:35.770 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-16T23:38:35.772 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-16T23:38:35.804 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-16T23:38:35.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-16T23:38:35.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-16T23:38:35.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-16T23:38:35.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-16T23:38:35.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-16T23:38:35.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-16T23:38:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-16T23:38:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-16T23:38:35.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-16T23:38:35.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-16T23:38:35.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-16T23:38:35.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-16T23:38:35.839 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-16T23:38:35.839 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-16T23:38:35.839 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-16T23:38:35.839 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-16T23:38:35.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-16T23:38:35.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-16T23:38:36.409 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-16T23:38:36.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-16T23:38:36.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-16T23:38:36.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-16T23:38:36.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-16T23:38:36.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-16T23:38:36.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-16T23:38:36.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-16T23:38:36.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-16T23:38:36.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-16T23:38:36.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-16T23:38:36.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-16T23:38:36.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-16T23:38:36.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-16T23:38:36.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-16T23:38:36.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-16T23:38:36.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-16T23:38:36.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-16T23:38:36.788 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-16T23:38:36.991 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-16T23:38:36.991 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-16T23:38:36.995 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-16T23:38:37.003 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-16T23:38:37.003 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-16T23:38:37.003 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-16T23:38:37.004 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-16T23:38:37.004 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-16T23:38:37.004 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-16T23:38:37.004 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-16T23:38:37.004 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-16T23:38:37.005 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-16T23:38:37.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-16T23:38:37.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-16T23:38:37.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-16T23:38:37.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-16T23:38:37.092 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-16T23:38:37.092 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-16T23:38:37.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-16T23:38:37.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-16T23:38:37.095 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-16T23:38:37.096 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-16T23:38:37.097 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-16T23:38:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-16T23:38:37.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-16T23:38:37.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-16T23:38:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-16T23:38:37.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-16T23:38:37.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-16T23:38:37.104 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-16T23:38:37.105 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-16T23:38:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-16T23:38:37.107 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-16T23:38:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-16T23:38:37.109 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-16T23:38:37.110 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-16T23:38:37.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-16T23:38:37.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-16T23:38:37.113 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-16T23:38:37.113 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-16T23:38:37.113 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-16T23:38:37.113 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-16T23:38:37.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-16T23:38:37.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-16T23:38:37.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-16T23:38:37.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-16T23:38:37.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-16T23:38:37.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-16T23:38:37.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-16T23:38:37.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-16T23:38:37.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-16T23:38:37.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-16T23:38:37.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-16T23:38:37.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-16T23:38:37.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-16T23:38:37.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-16T23:38:37.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-16T23:38:37.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-16T23:38:37.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-16T23:38:37.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-16T23:38:37.163 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-16T23:38:37.164 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-16T23:38:37.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-16T23:38:37.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-16T23:38:37.248 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-16T23:38:37.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-16T23:38:37.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-16T23:38:37.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-16T23:38:37.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-16T23:38:37.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-16T23:38:37.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-16T23:38:37.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-16T23:38:37.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-16T23:38:37.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-16T23:38:37.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-16T23:38:37.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-16T23:38:37.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-16T23:38:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-16T23:38:37.596 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-16T23:38:37.597 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-16T23:38:37.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-16T23:38:37.863 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-16T23:38:37.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-16T23:38:37.930 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-16T23:38:38.006 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-16T23:38:38.007 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-16T23:38:38.009 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-16T23:38:38.010 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-16T23:38:38.010 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-16T23:38:38.010 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-16T23:38:38.010 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-16T23:38:38.010 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-16T23:38:38.033 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-16T23:38:38.033 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-16T23:38:38.033 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-16T23:38:38.034 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-16T23:38:38.035 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-16T23:38:38.036 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-16T23:38:38.036 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-16T23:38:38.036 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-16T23:38:38.048 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-16T23:38:38.049 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-16T23:38:38.050 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-16T23:38:38.051 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-16T23:38:38.051 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-16T23:38:38.051 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-16T23:38:38.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-16T23:38:38.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-16T23:38:38.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-16T23:38:38.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-16T23:38:38.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-16T23:38:38.434 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-16T23:38:38.434 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-16T23:38:38.434 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-16T23:38:38.434 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-16T23:38:38.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-16T23:38:38.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-16T23:38:38.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-16T23:38:38.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-16T23:38:38.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-16T23:38:38.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-16T23:38:38.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-16T23:38:38.575 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-16T23:38:38.648 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-16T23:38:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-16T23:38:38.650 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-16T23:38:38.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-16T23:38:38.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-16T23:38:38.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-16T23:38:38.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-16T23:38:38.651 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-16T23:38:38.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-16T23:38:38.697 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-16T23:38:38.698 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-16T23:38:38.699 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-16T23:38:38.700 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-16T23:38:38.742 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-16T23:38:38.743 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-16T23:38:38.744 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-16T23:38:38.745 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-16T23:38:38.746 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-16T23:38:38.747 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-16T23:38:38.748 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-16T23:38:38.749 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-16T23:38:38.750 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-16T23:38:38.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-16T23:38:38.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-16T23:38:38.751 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-16T23:38:38.752 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-16T23:38:38.753 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-16T23:38:38.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-16T23:38:38.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-16T23:38:38.754 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-16T23:38:38.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-16T23:38:38.755 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-16T23:38:38.756 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-16T23:38:38.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-16T23:38:38.757 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-16T23:38:38.758 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-16T23:38:38.759 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-16T23:38:38.761 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-16T23:38:38.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-16T23:38:38.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-16T23:38:38.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-16T23:38:38.762 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-16T23:38:38.764 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-16T23:38:38.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-16T23:38:38.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-16T23:38:38.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-16T23:38:38.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-16T23:38:38.765 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-16T23:38:38.766 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-16T23:38:38.767 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-16T23:38:38.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-16T23:38:38.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-16T23:38:38.768 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-16T23:38:38.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-16T23:38:38.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-16T23:38:38.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-16T23:38:38.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-16T23:38:38.817 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-16T23:38:38.818 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-16T23:38:38.819 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-16T23:38:38.820 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-16T23:38:38.821 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-16T23:38:38.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-16T23:38:38.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-16T23:38:38.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-16T23:38:38.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-16T23:38:38.822 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-16T23:38:38.824 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-16T23:38:38.825 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-16T23:38:38.826 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-16T23:38:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-16T23:38:38.828 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-16T23:38:38.829 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-16T23:38:38.830 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-16T23:38:38.831 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-16T23:38:38.832 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-16T23:38:38.833 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-16T23:38:38.835 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-16T23:38:38.836 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-16T23:38:38.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-16T23:38:38.837 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-16T23:38:38.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-16T23:38:38.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-16T23:38:38.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-16T23:38:38.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-16T23:38:38.838 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-16T23:38:38.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-16T23:38:38.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-16T23:38:38.864 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-16T23:38:38.865 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-16T23:38:38.866 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-16T23:38:38.867 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-16T23:38:38.868 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-16T23:38:38.869 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-16T23:38:38.870 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-16T23:38:38.871 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-16T23:38:38.872 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-16T23:38:38.873 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-16T23:38:38.874 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-16T23:38:38.875 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-16T23:38:38.876 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-16T23:38:38.877 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-16T23:38:38.878 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-16T23:38:38.879 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-16T23:38:38.880 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-16T23:38:38.881 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-16T23:38:38.881 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-16T23:38:38.881 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-16T23:38:38.881 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-16T23:38:38.893 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-16T23:38:38.894 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-16T23:38:38.895 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-16T23:38:38.896 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-16T23:38:38.897 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-16T23:38:38.898 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-16T23:38:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-16T23:38:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-16T23:38:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-16T23:38:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-16T23:38:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-16T23:38:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-16T23:38:38.901 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-16T23:38:38.902 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-16T23:38:38.902 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-16T23:38:38.902 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-16T23:38:38.902 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-16T23:38:38.904 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-16T23:38:38.905 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-16T23:38:38.906 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-16T23:38:38.907 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-16T23:38:38.908 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-16T23:38:38.909 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-16T23:38:38.910 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-16T23:38:38.911 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-16T23:38:38.912 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:38.913 DEBUG:teuthology.orchestra.run.smithi006:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-16T23:38:38.941 INFO:teuthology.orchestra.run.smithi006.stdout:postinstall scriptlet (using /bin/sh): 2023-11-16T23:38:38.941 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-16T23:38:38.941 INFO:teuthology.orchestra.run.smithi006.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T23:38:38.941 INFO:teuthology.orchestra.run.smithi006.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T23:38:38.941 INFO:teuthology.orchestra.run.smithi006.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:38.942 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-16T23:38:38.943 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2023-11-16T23:38:39.022 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.02 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 123.el8 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Group : System Environment/Base 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 9265450 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-16T23:38:39.023 INFO:teuthology.orchestra.run.smithi006.stdout:Relocations : (not relocatable) 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : Red Hat, Inc. 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : Red Hat, Inc. 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:38:39.024 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2023-11-16T23:38:39.025 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-16T23:38:39.025 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:39.026 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-16T23:38:41.964 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2023-11-16T23:38:44.162 INFO:teuthology.orchestra.run.smithi006.stderr:done 2023-11-16T23:38:44.165 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-11-16T23:38:44.184 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.kqIc9hyCuD 2023-11-16T23:38:44.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kqIc9hyCuD 2023-11-16T23:38:44.255 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.kqIc9hyCuD 2023-11-16T23:38:44.399 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.kqIc9hyCuD is 6KB 2023-11-16T23:38:44.447 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.kqIc9hyCuD 2023-11-16T23:38:44.463 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2023-11-16T23:38:44.535 INFO:teuthology.orchestra.run.smithi006.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-16T23:38:44.536 INFO:teuthology.orchestra.run.smithi006.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-16T23:38:44.536 INFO:teuthology.orchestra.run.smithi006.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-16T23:38:44.537 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:38:44.692 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2023-11-16T23:39:14.696 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-16T23:39:14.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:39:14.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:39:33.114 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-11-16T23:39:42.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:39:42.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:40:42.118 DEBUG:teuthology.orchestra.remote:timed out 2023-11-16T23:40:54.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:40:54.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:40:54.627 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-11-16T23:40:55.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:40:55.077 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-16T23:40:55.078 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-11-16T23:40:55.121 INFO:teuthology.orchestra.run.smithi006.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:40:55.121 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-16T23:40:55.121 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-16T23:40:55.121 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-16T23:40:56.122 DEBUG:teuthology.parallel:result is None 2023-11-16T23:41:04.132 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-16T23:41:04.143 INFO:teuthology.task.internal:Creating test directory... 2023-11-16T23:41:04.143 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T23:41:04.148 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T23:41:04.151 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T23:41:04.191 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-16T23:41:04.203 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-16T23:41:04.214 INFO:teuthology.task.internal:Creating archive directory... 2023-11-16T23:41:04.214 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T23:41:04.220 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T23:41:04.222 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T23:41:04.277 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-16T23:41:04.612 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-16T23:41:04.612 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T23:41:04.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:41:04.631 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T23:41:04.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:41:04.681 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T23:41:04.685 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T23:41:04.688 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T23:41:04.767 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.778 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.787 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.797 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.826 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.885 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T23:41:04.888 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-16T23:41:04.896 INFO:teuthology.task.internal:Configuring sudo... 2023-11-16T23:41:04.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T23:41:04.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T23:41:04.904 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T23:41:04.968 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-16T23:41:04.980 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-16T23:41:04.980 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T23:41:04.994 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T23:41:04.996 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T23:41:05.032 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.211 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.282 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.352 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:41:05.352 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T23:41:05.421 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.531 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.575 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.620 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:41:05.621 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T23:41:05.690 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.738 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T23:41:05.796 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.883 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T23:41:05.928 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:41:05.928 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T23:41:06.024 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2023-11-16T23:41:06.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2023-11-16T23:41:06.031 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-11-16T23:41:06.096 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T23:41:06.123 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T23:41:06.140 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T23:41:06.509 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-16T23:41:06.517 INFO:teuthology.task.internal:Starting timer... 2023-11-16T23:41:06.517 INFO:teuthology.run_tasks:Running task pcp... 2023-11-16T23:41:06.542 INFO:teuthology.run_tasks:Running task selinux... 2023-11-16T23:41:06.553 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-16T23:41:06.553 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2023-11-16T23:41:06.674 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2023-11-16T23:41:06.677 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2023-11-16T23:41:06.744 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2023-11-16T23:41:06.746 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-11-16T23:41:06.811 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-11-16T23:41:06.813 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-16T23:41:06.813 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2023-11-16T23:41:06.868 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2023-11-16T23:41:06.868 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2023-11-16T23:41:06.889 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2023-11-16T23:41:06.890 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-11-16T23:41:06.918 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-11-16T23:41:06.919 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-11-16T23:41:06.919 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T23:41:06.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:41:06.993 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T23:41:07.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:41:07.020 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T23:41:07.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T23:41:07.049 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-16T23:41:07.049 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2023-11-16T23:41:07.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2023-11-16T23:41:07.146 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-11-16T23:41:07.174 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-16T23:41:07.184 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-16T23:41:07.190 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-16T23:41:07.365 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-16T23:41:07.384 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-16T23:41:07.385 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi087.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-16T23:54:00.997 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-11-16T23:54:00.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:54:00.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:54:01.093 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-11-16T23:54:01.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-16T23:54:01.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-11-16T23:54:01.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:54:01.277 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-11-16T23:54:01.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-11-16T23:54:01.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-16T23:54:01.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T23:54:01.417 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-11-16T23:54:01.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-16T23:54:01.490 INFO:teuthology.run_tasks:Running task clock... 2023-11-16T23:54:01.500 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-16T23:54:01.500 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T23:54:01.501 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T23:54:01.505 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T23:54:01.505 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T23:54:01.516 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T23:54:01.516 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T23:54:01.541 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T23:54:01.562 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T23:54:01.565 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T23:54:01.578 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T23:54:01.580 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T23:54:01.613 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T23:54:01.626 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2023-11-16T23:54:01.642 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2023-11-16T23:54:01.654 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T23:54:01.661 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-11-16T23:54:01.673 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-11-16T23:54:01.673 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T23:54:01.688 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T23:54:01.702 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T23:54:01.706 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2023-11-16T23:54:01.736 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2023-11-16T23:54:01.743 INFO:teuthology.orchestra.run.smithi087.stderr:bash: ntpq: command not found 2023-11-16T23:54:01.747 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T23:54:01.747 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2023-11-16T23:54:01.747 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.747 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.747 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.748 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.772 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T23:54:01.776 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.782 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.806 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T23:54:01.907 INFO:teuthology.orchestra.run.smithi006.stderr:bash: ntpq: command not found 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.915 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T23:54:01.917 INFO:teuthology.run_tasks:Running task pexec... 2023-11-16T23:54:01.926 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-16T23:54:01.926 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi006.front.sepia.ceph.com 2023-11-16T23:54:01.927 DEBUG:teuthology.orchestra.run.smithi006:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T23:54:01.927 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi087.front.sepia.ceph.com 2023-11-16T23:54:01.927 DEBUG:teuthology.orchestra.run.smithi087:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T23:54:01.927 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi186.front.sepia.ceph.com 2023-11-16T23:54:01.928 DEBUG:teuthology.orchestra.run.smithi186:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T23:54:02.298 INFO:teuthology.orchestra.run.smithi087.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:02.303 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:02.441 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:03.123 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:05:07 ago on Thu 16 Nov 2023 11:48:55 PM UTC. 2023-11-16T23:54:03.147 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:06 ago on Thu 16 Nov 2023 11:48:56 PM UTC. 2023-11-16T23:54:03.440 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:50 ago on Thu 16 Nov 2023 11:52:12 PM UTC. 2023-11-16T23:54:04.838 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-11-16T23:54:04.839 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-11-16T23:54:04.839 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout:Resetting modules: 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2023-11-16T23:54:04.840 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:05.018 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-16T23:54:05.019 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-11-16T23:54:05.019 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-11-16T23:54:05.019 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-11-16T23:54:05.019 INFO:teuthology.orchestra.run.smithi186.stdout:Resetting modules: 2023-11-16T23:54:05.019 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools 2023-11-16T23:54:05.020 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:05.020 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-11-16T23:54:05.020 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-11-16T23:54:05.020 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:05.447 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-16T23:54:05.448 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-16T23:54:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2023-11-16T23:54:05.448 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-16T23:54:05.448 INFO:teuthology.orchestra.run.smithi006.stdout:Resetting modules: 2023-11-16T23:54:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: container-tools 2023-11-16T23:54:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:05.449 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-16T23:54:05.449 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-16T23:54:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:05.807 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:05:10 ago on Thu 16 Nov 2023 11:48:55 PM UTC. 2023-11-16T23:54:05.993 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:09 ago on Thu 16 Nov 2023 11:48:56 PM UTC. 2023-11-16T23:54:06.781 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:54 ago on Thu 16 Nov 2023 11:52:12 PM UTC. 2023-11-16T23:54:08.339 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-11-16T23:54:08.564 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-16T23:54:08.801 INFO:teuthology.orchestra.run.smithi087.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:09.038 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:09.393 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-16T23:54:09.598 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:05:14 ago on Thu 16 Nov 2023 11:48:55 PM UTC. 2023-11-16T23:54:09.894 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:13 ago on Thu 16 Nov 2023 11:48:56 PM UTC. 2023-11-16T23:54:10.013 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:11.073 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:58 ago on Thu 16 Nov 2023 11:52:12 PM UTC. 2023-11-16T23:54:11.339 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-11-16T23:54:11.429 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================================ 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================================ 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout:Installing group/module packages: 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T23:54:11.430 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout:Installing module profiles: 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools/common 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling module streams: 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: container-tools rhel8 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================================ 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout:Install 10 Packages 2023-11-16T23:54:11.431 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:11.432 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 18 M 2023-11-16T23:54:11.432 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 65 M 2023-11-16T23:54:11.432 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-11-16T23:54:11.497 INFO:teuthology.orchestra.run.smithi087.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 627 kB/s | 41 kB 00:00 2023-11-16T23:54:11.522 INFO:teuthology.orchestra.run.smithi087.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 275 kB/s | 25 kB 00:00 2023-11-16T23:54:11.548 INFO:teuthology.orchestra.run.smithi087.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 982 kB/s | 49 kB 00:00 2023-11-16T23:54:11.565 INFO:teuthology.orchestra.run.smithi087.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 708 kB/s | 94 kB 00:00 2023-11-16T23:54:11.598 INFO:teuthology.orchestra.run.smithi087.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-11-16T23:54:11.625 INFO:teuthology.orchestra.run.smithi087.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-11-16T23:54:11.632 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-16T23:54:11.650 INFO:teuthology.orchestra.run.smithi087.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout:Installing group/module packages: 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T23:54:11.729 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout:Installing module profiles: 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools/common 2023-11-16T23:54:11.730 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools rhel8 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout:Install 10 Packages 2023-11-16T23:54:11.731 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:11.732 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 18 M 2023-11-16T23:54:11.732 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 65 M 2023-11-16T23:54:11.732 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-11-16T23:54:11.741 INFO:teuthology.orchestra.run.smithi087.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-11-16T23:54:11.823 INFO:teuthology.orchestra.run.smithi186.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 447 kB/s | 41 kB 00:00 2023-11-16T23:54:11.848 INFO:teuthology.orchestra.run.smithi186.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 213 kB/s | 25 kB 00:00 2023-11-16T23:54:11.849 INFO:teuthology.orchestra.run.smithi087.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-11-16T23:54:11.882 INFO:teuthology.orchestra.run.smithi186.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 842 kB/s | 49 kB 00:00 2023-11-16T23:54:11.907 INFO:teuthology.orchestra.run.smithi186.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 535 kB/s | 94 kB 00:00 2023-11-16T23:54:11.933 INFO:teuthology.orchestra.run.smithi087.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 21 MB/s | 8.0 MB 00:00 2023-11-16T23:54:11.933 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:54:11.933 INFO:teuthology.orchestra.run.smithi087.stdout:Total 36 MB/s | 18 MB 00:00 2023-11-16T23:54:11.941 INFO:teuthology.orchestra.run.smithi186.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-11-16T23:54:11.957 INFO:teuthology.orchestra.run.smithi186.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-11-16T23:54:11.983 INFO:teuthology.orchestra.run.smithi186.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-11-16T23:54:12.119 INFO:teuthology.orchestra.run.smithi186.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-11-16T23:54:12.133 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-11-16T23:54:12.151 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-11-16T23:54:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-11-16T23:54:12.242 INFO:teuthology.orchestra.run.smithi186.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 25 MB/s | 7.0 MB 00:00 2023-11-16T23:54:12.333 INFO:teuthology.orchestra.run.smithi186.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-11-16T23:54:12.334 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-11-16T23:54:12.334 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:54:12.334 INFO:teuthology.orchestra.run.smithi186.stdout:Total 30 MB/s | 18 MB 00:00 2023-11-16T23:54:12.337 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-11-16T23:54:12.535 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-11-16T23:54:12.554 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-11-16T23:54:12.554 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-11-16T23:54:12.737 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-11-16T23:54:12.741 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-11-16T23:54:12.820 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-11-16T23:54:13.049 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T23:54:13.138 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-16T23:54:13.236 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:13.245 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================================ 2023-11-16T23:54:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-11-16T23:54:13.245 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================================ 2023-11-16T23:54:13.245 INFO:teuthology.orchestra.run.smithi006.stdout:Installing group/module packages: 2023-11-16T23:54:13.245 INFO:teuthology.orchestra.run.smithi006.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout:Installing module profiles: 2023-11-16T23:54:13.246 INFO:teuthology.orchestra.run.smithi006.stdout: container-tools/common 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling module streams: 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: container-tools rhel8 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================================ 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout:Install 10 Packages 2023-11-16T23:54:13.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:13.248 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 18 M 2023-11-16T23:54:13.248 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 65 M 2023-11-16T23:54:13.248 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-11-16T23:54:13.298 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-11-16T23:54:13.317 INFO:teuthology.orchestra.run.smithi006.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-11-16T23:54:13.342 INFO:teuthology.orchestra.run.smithi006.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 267 kB/s | 25 kB 00:00 2023-11-16T23:54:13.376 INFO:teuthology.orchestra.run.smithi006.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 321 kB/s | 41 kB 00:00 2023-11-16T23:54:13.418 INFO:teuthology.orchestra.run.smithi006.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 490 kB/s | 49 kB 00:00 2023-11-16T23:54:13.446 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T23:54:13.500 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T23:54:13.577 INFO:teuthology.orchestra.run.smithi006.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.2 MB/s | 524 kB 00:00 2023-11-16T23:54:13.602 INFO:teuthology.orchestra.run.smithi006.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 813 kB/s | 149 kB 00:00 2023-11-16T23:54:13.636 INFO:teuthology.orchestra.run.smithi006.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-11-16T23:54:13.658 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T23:54:13.678 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:13.761 INFO:teuthology.orchestra.run.smithi006.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-11-16T23:54:13.911 INFO:teuthology.orchestra.run.smithi006.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 23 MB/s | 7.0 MB 00:00 2023-11-16T23:54:13.915 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T23:54:14.003 INFO:teuthology.orchestra.run.smithi006.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 13 MB/s | 8.0 MB 00:00 2023-11-16T23:54:14.004 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:54:14.004 INFO:teuthology.orchestra.run.smithi006.stdout:Total 24 MB/s | 18 MB 00:00 2023-11-16T23:54:14.039 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T23:54:14.147 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T23:54:14.352 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-11-16T23:54:14.377 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-11-16T23:54:14.377 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-11-16T23:54:14.499 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T23:54:14.792 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T23:54:14.916 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-11-16T23:54:14.921 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-11-16T23:54:15.262 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T23:54:15.460 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-11-16T23:54:15.648 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T23:54:15.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T23:54:15.808 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:15.874 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T23:54:16.052 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T23:54:16.070 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T23:54:16.228 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:16.290 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T23:54:16.308 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T23:54:16.525 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:16.525 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T23:54:16.526 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T23:54:16.533 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T23:54:16.706 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T23:54:16.727 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T23:54:16.920 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:17.173 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T23:54:17.174 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T23:54:17.570 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T23:54:17.710 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T23:54:17.710 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:05:22 ago on Thu 16 Nov 2023 11:48:55 PM UTC. 2023-11-16T23:54:18.343 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T23:54:18.344 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:05:21 ago on Thu 16 Nov 2023 11:48:56 PM UTC. 2023-11-16T23:54:18.549 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T23:54:18.814 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T23:54:19.038 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T23:54:19.265 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:19.659 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T23:54:19.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T23:54:20.458 INFO:teuthology.orchestra.run.smithi087.stdout:Installed products updated. 2023-11-16T23:54:20.668 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:20.668 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-11-16T23:54:20.668 INFO:teuthology.orchestra.run.smithi087.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:54:20.669 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-11-16T23:54:20.851 DEBUG:teuthology.parallel:result is None 2023-11-16T23:54:21.012 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-11-16T23:54:21.068 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T23:54:21.069 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:08 ago on Thu 16 Nov 2023 11:52:12 PM UTC. 2023-11-16T23:54:21.270 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:21.270 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:21.271 INFO:teuthology.orchestra.run.smithi186.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:21.272 INFO:teuthology.orchestra.run.smithi186.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T23:54:21.272 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:54:21.272 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-16T23:54:21.516 DEBUG:teuthology.parallel:result is None 2023-11-16T23:54:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:Installed products updated. 2023-11-16T23:54:24.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:24.156 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-11-16T23:54:24.156 INFO:teuthology.orchestra.run.smithi006.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:54:24.157 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-16T23:54:24.361 DEBUG:teuthology.parallel:result is None 2023-11-16T23:54:24.361 INFO:teuthology.run_tasks:Running task install... 2023-11-16T23:54:24.372 DEBUG:teuthology.task.install:project ceph 2023-11-16T23:54:24.372 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'}} 2023-11-16T23:54:24.373 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'} 2023-11-16T23:54:24.373 INFO:teuthology.task.install:Using flavor: default 2023-11-16T23:54:24.378 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-16T23:54:24.378 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-16T23:54:24.379 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-16T23:54:24.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:54:24.380 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-16T23:54:24.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:54:24.381 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-16T23:54:24.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:54:24.568 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-16T23:54:24.569 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-16T23:54:24.576 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-16T23:54:24.576 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-16T23:54:24.691 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-16T23:54:24.691 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-16T23:54:24.750 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T23:54:24.750 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:54:24.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T23:54:24.757 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T23:54:24.757 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:54:24.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T23:54:24.784 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T23:54:24.784 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T23:54:24.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T23:54:24.821 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T23:54:24.850 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T23:54:24.870 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T23:54:24.870 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:54:24.870 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T23:54:24.896 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T23:54:24.897 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T23:54:24.930 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T23:54:24.931 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T23:54:24.964 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T23:54:24.995 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2023-11-16T23:54:24.997 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2023-11-16T23:54:25.029 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T23:54:25.042 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T23:54:25.098 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2023-11-16T23:54:25.102 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2023-11-16T23:54:25.109 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-11-16T23:54:25.110 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-11-16T23:54:25.391 INFO:teuthology.orchestra.run.smithi087.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:25.524 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:25.580 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:35.359 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-11-16T23:54:35.359 INFO:teuthology.orchestra.run.smithi087.stdout:44 files removed 2023-11-16T23:54:35.359 INFO:teuthology.orchestra.run.smithi006.stdout:44 files removed 2023-11-16T23:54:35.359 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T23:54:35.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T23:54:35.360 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T23:54:35.706 INFO:teuthology.orchestra.run.smithi087.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:35.903 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:35.903 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-11-16T23:54:36.650 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2023-11-16T23:54:36.657 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 180 kB/s | 80 kB 00:00 2023-11-16T23:54:36.664 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 266 kB/s | 80 kB 00:00 2023-11-16T23:54:36.909 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 41 kB/s | 9.4 kB 00:00 2023-11-16T23:54:36.931 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 41 kB/s | 9.4 kB 00:00 2023-11-16T23:54:37.008 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-11-16T23:54:37.307 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-11-16T23:54:37.318 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-11-16T23:54:37.367 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-11-16T23:54:37.450 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-11-16T23:54:37.460 INFO:teuthology.orchestra.run.smithi087.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-16T23:54:37.509 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-16T23:54:38.361 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-11-16T23:54:38.476 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-11-16T23:54:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-11-16T23:54:41.937 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 355 kB/s | 24 kB 00:00 2023-11-16T23:54:42.193 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 562 kB/s | 24 kB 00:00 2023-11-16T23:54:42.871 INFO:teuthology.orchestra.run.smithi087.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-11-16T23:54:42.901 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 272 kB/s | 24 kB 00:00 2023-11-16T23:54:43.036 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-11-16T23:54:44.020 INFO:teuthology.orchestra.run.smithi006.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 53 MB 00:01 2023-11-16T23:54:51.023 INFO:teuthology.orchestra.run.smithi087.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-11-16T23:54:51.047 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-11-16T23:54:53.163 INFO:teuthology.orchestra.run.smithi006.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-11-16T23:55:02.023 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:02.023 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:02.038 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.039 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.040 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T23:55:02.042 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T23:55:02.044 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T23:55:02.047 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.048 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T23:55:02.136 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:02.137 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:02.151 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.152 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.154 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T23:55:02.155 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T23:55:02.157 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T23:55:02.160 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T23:55:02.161 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T23:55:02.182 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================================================== 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================================================== 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T23:55:02.196 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-16T23:55:02.197 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-16T23:55:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-16T23:55:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-16T23:55:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-16T23:55:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-16T23:55:02.198 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T23:55:02.199 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-16T23:55:02.200 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-16T23:55:02.201 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T23:55:02.202 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-16T23:55:02.203 INFO:teuthology.orchestra.run.smithi087.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T23:55:02.204 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T23:55:02.205 INFO:teuthology.orchestra.run.smithi087.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T23:55:02.206 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T23:55:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T23:55:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T23:55:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T23:55:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T23:55:02.207 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T23:55:02.208 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T23:55:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T23:55:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T23:55:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T23:55:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T23:55:02.209 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T23:55:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T23:55:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T23:55:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T23:55:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T23:55:02.210 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T23:55:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T23:55:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T23:55:02.211 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T23:55:02.212 INFO:teuthology.orchestra.run.smithi087.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T23:55:02.213 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling module streams: 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql 10 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout:=========================================================================================================================== 2023-11-16T23:55:02.214 INFO:teuthology.orchestra.run.smithi087.stdout:Install 128 Packages 2023-11-16T23:55:02.215 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 3 Packages 2023-11-16T23:55:02.215 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:55:02.216 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 197 M 2023-11-16T23:55:02.216 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2023-11-16T23:55:02.294 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-11-16T23:55:02.306 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-16T23:55:02.306 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-11-16T23:55:02.306 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-16T23:55:02.307 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T23:55:02.308 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T23:55:02.309 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-16T23:55:02.310 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T23:55:02.311 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-16T23:55:02.312 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-16T23:55:02.313 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T23:55:02.314 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T23:55:02.315 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-16T23:55:02.316 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T23:55:02.317 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T23:55:02.318 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T23:55:02.319 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T23:55:02.320 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T23:55:02.321 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-11-16T23:55:02.322 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:55:02.323 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 197 M 2023-11-16T23:55:02.323 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-11-16T23:55:02.507 INFO:teuthology.orchestra.run.smithi087.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-11-16T23:55:02.697 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-11-16T23:55:02.707 INFO:teuthology.orchestra.run.smithi087.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 4.1 MB/s | 848 kB 00:00 2023-11-16T23:55:02.758 INFO:teuthology.orchestra.run.smithi087.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.2 MB/s | 165 kB 00:00 2023-11-16T23:55:03.022 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 2.5 MB/s | 848 kB 00:00 2023-11-16T23:55:03.164 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 1.1 MB/s | 165 kB 00:00 2023-11-16T23:55:03.233 INFO:teuthology.orchestra.run.smithi087.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 4.8 MB/s | 2.3 MB 00:00 2023-11-16T23:55:03.300 INFO:teuthology.orchestra.run.smithi087.stdout:(5/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 5.8 MB/s | 5.3 MB 00:00 2023-11-16T23:55:03.625 INFO:teuthology.orchestra.run.smithi087.stdout:(6/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 4.0 MB/s | 1.5 MB 00:00 2023-11-16T23:55:03.639 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 4.9 MB/s | 5.3 MB 00:01 2023-11-16T23:55:03.799 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 3.6 MB/s | 2.3 MB 00:00 2023-11-16T23:55:04.050 INFO:teuthology.orchestra.run.smithi087.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 6.1 MB/s | 4.6 MB 00:00 2023-11-16T23:55:04.157 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 1.5 MB 00:00 2023-11-16T23:55:04.594 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 5.8 MB/s | 4.6 MB 00:00 2023-11-16T23:55:05.549 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:05.550 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T23:55:05.566 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:05.567 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T23:55:05.568 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T23:55:05.569 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T23:55:05.572 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T23:55:05.575 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T23:55:05.577 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T23:55:05.727 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-16T23:55:05.740 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-16T23:55:05.741 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-16T23:55:05.742 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T23:55:05.743 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2023-11-16T23:55:05.744 INFO:teuthology.orchestra.run.smithi006.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-16T23:55:05.745 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T23:55:05.746 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T23:55:05.747 INFO:teuthology.orchestra.run.smithi006.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T23:55:05.748 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T23:55:05.749 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T23:55:05.750 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T23:55:05.752 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T23:55:05.752 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T23:55:05.753 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T23:55:05.754 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T23:55:05.754 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T23:55:05.754 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T23:55:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T23:55:05.756 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T23:55:05.756 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T23:55:05.756 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T23:55:05.756 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T23:55:05.756 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T23:55:05.757 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T23:55:05.757 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T23:55:05.757 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T23:55:05.757 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T23:55:05.757 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling module streams: 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql 10 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:55:05.758 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-16T23:55:05.759 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-16T23:55:05.759 INFO:teuthology.orchestra.run.smithi006.stdout:Install 128 Packages 2023-11-16T23:55:05.759 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 3 Packages 2023-11-16T23:55:05.759 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:55:05.760 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 197 M 2023-11-16T23:55:05.760 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-11-16T23:55:06.230 INFO:teuthology.orchestra.run.smithi006.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-11-16T23:55:06.316 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 7.5 MB/s | 13 MB 00:01 2023-11-16T23:55:06.532 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-common-18.0.0-3906.g1c956514.el8. 5.4 MB/s | 22 MB 00:03 2023-11-16T23:55:06.557 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-selinux-18.0.0-3906.g1c956514.el 101 kB/s | 24 kB 00:00 2023-11-16T23:55:06.697 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.8 MB/s | 258 kB 00:00 2023-11-16T23:55:06.791 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 331 kB/s | 31 kB 00:00 2023-11-16T23:55:06.901 INFO:teuthology.orchestra.run.smithi087.stdout:(8/131): ceph-osd-18.0.0-3906.g1c956514.el8.x86 5.4 MB/s | 18 MB 00:03 2023-11-16T23:55:06.966 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 4.3 MB/s | 771 kB 00:00 2023-11-16T23:55:06.992 INFO:teuthology.orchestra.run.smithi087.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 266 kB/s | 24 kB 00:00 2023-11-16T23:55:06.997 INFO:teuthology.orchestra.run.smithi006.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 1.1 MB/s | 848 kB 00:00 2023-11-16T23:55:07.058 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.5 MB/s | 137 kB 00:00 2023-11-16T23:55:07.158 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 1.4 MB/s | 139 kB 00:00 2023-11-16T23:55:07.197 INFO:teuthology.orchestra.run.smithi006.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 828 kB/s | 165 kB 00:00 2023-11-16T23:55:07.542 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 5.2 MB/s | 18 MB 00:03 2023-11-16T23:55:07.740 INFO:teuthology.orchestra.run.smithi006.stdout:(4/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 2.8 MB/s | 5.3 MB 00:01 2023-11-16T23:55:07.768 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): libradosstriper1-18.0.0-3906.g1c95651 777 kB/s | 473 kB 00:00 2023-11-16T23:55:07.860 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 489 kB/s | 45 kB 00:00 2023-11-16T23:55:07.952 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.3 MB/s | 119 kB 00:00 2023-11-16T23:55:08.045 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 2.1 MB/s | 197 kB 00:00 2023-11-16T23:55:08.131 INFO:teuthology.orchestra.run.smithi006.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 4.0 MB/s | 1.5 MB 00:00 2023-11-16T23:55:08.136 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 4.0 MB/s | 371 kB 00:00 2023-11-16T23:55:08.228 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 3.9 MB/s | 364 kB 00:00 2023-11-16T23:55:08.336 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 987 kB/s | 107 kB 00:00 2023-11-16T23:55:08.420 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.1 MB/s | 94 kB 00:00 2023-11-16T23:55:09.078 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 5.2 MB/s | 3.4 MB 00:00 2023-11-16T23:55:09.183 INFO:teuthology.orchestra.run.smithi006.stdout:(6/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-11-16T23:55:09.184 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-16T23:55:09.283 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 221 kB/s | 23 kB 00:00 2023-11-16T23:55:09.387 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.2 MB/s | 127 kB 00:00 2023-11-16T23:55:09.440 INFO:teuthology.orchestra.run.smithi006.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 3.5 MB/s | 4.6 MB 00:01 2023-11-16T23:55:09.718 INFO:teuthology.orchestra.run.smithi087.stdout:(10/131): ceph-radosgw-18.0.0-3906.g1c956514.el 2.3 MB/s | 13 MB 00:05 2023-11-16T23:55:09.771 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 4.5 MB/s | 1.7 MB 00:00 2023-11-16T23:55:09.876 INFO:teuthology.orchestra.run.smithi087.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.6 MB/s | 258 kB 00:00 2023-11-16T23:55:09.964 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): librgw2-18.0.0-3906.g1c956514.el8.x86 2.7 MB/s | 6.5 MB 00:02 2023-11-16T23:55:09.977 INFO:teuthology.orchestra.run.smithi087.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 309 kB/s | 31 kB 00:00 2023-11-16T23:55:10.046 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 2.8 MB/s | 240 kB 00:00 2023-11-16T23:55:10.130 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 568 kB/s | 47 kB 00:00 2023-11-16T23:55:10.214 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3906.g1 174 kB/s | 15 kB 00:00 2023-11-16T23:55:10.322 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 2.4 MB/s | 265 kB 00:00 2023-11-16T23:55:10.423 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 836 kB/s | 83 kB 00:00 2023-11-16T23:55:10.515 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 196 kB/s | 18 kB 00:00 2023-11-16T23:55:10.527 INFO:teuthology.orchestra.run.smithi087.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 1.4 MB/s | 771 kB 00:00 2023-11-16T23:55:10.852 INFO:teuthology.orchestra.run.smithi087.stdout:(14/131): ceph-common-18.0.0-3906.g1c956514.el8 2.5 MB/s | 22 MB 00:08 2023-11-16T23:55:10.877 INFO:teuthology.orchestra.run.smithi087.stdout:(15/131): libcephsqlite-18.0.0-3906.g1c956514.e 392 kB/s | 137 kB 00:00 2023-11-16T23:55:11.036 INFO:teuthology.orchestra.run.smithi087.stdout:(16/131): librados-devel-18.0.0-3906.g1c956514. 759 kB/s | 139 kB 00:00 2023-11-16T23:55:11.098 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 525 kB/s | 306 kB 00:00 2023-11-16T23:55:11.228 INFO:teuthology.orchestra.run.smithi087.stdout:(17/131): libradosstriper1-18.0.0-3906.g1c95651 1.3 MB/s | 473 kB 00:00 2023-11-16T23:55:11.232 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2023-11-16T23:55:11.303 INFO:teuthology.orchestra.run.smithi087.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 597 kB/s | 45 kB 00:00 2023-11-16T23:55:11.391 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 4.6 MB/s | 7.4 MB 00:01 2023-11-16T23:55:11.415 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 409 kB/s | 75 kB 00:00 2023-11-16T23:55:11.437 INFO:teuthology.orchestra.run.smithi087.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 890 kB/s | 119 kB 00:00 2023-11-16T23:55:11.554 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 316 kB/s | 44 kB 00:00 2023-11-16T23:55:11.579 INFO:teuthology.orchestra.run.smithi087.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 1.4 MB/s | 197 kB 00:00 2023-11-16T23:55:11.666 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-11-16T23:55:11.766 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-11-16T23:55:11.787 INFO:teuthology.orchestra.run.smithi087.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 1.7 MB/s | 371 kB 00:00 2023-11-16T23:55:11.884 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 140 kB/s | 16 kB 00:00 2023-11-16T23:55:11.984 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-11-16T23:55:11.988 INFO:teuthology.orchestra.run.smithi087.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 1.8 MB/s | 364 kB 00:00 2023-11-16T23:55:12.043 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 730 kB/s | 476 kB 00:00 2023-11-16T23:55:12.071 INFO:teuthology.orchestra.run.smithi087.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.3 MB/s | 107 kB 00:00 2023-11-16T23:55:12.143 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 814 kB/s | 81 kB 00:00 2023-11-16T23:55:12.154 INFO:teuthology.orchestra.run.smithi087.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.1 MB/s | 94 kB 00:00 2023-11-16T23:55:12.210 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-11-16T23:55:12.243 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-11-16T23:55:12.311 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-11-16T23:55:12.315 INFO:teuthology.orchestra.run.smithi006.stdout:(8/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 4.5 MB/s | 13 MB 00:02 2023-11-16T23:55:12.382 INFO:teuthology.orchestra.run.smithi006.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 365 kB/s | 24 kB 00:00 2023-11-16T23:55:12.412 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 488 kB/s | 49 kB 00:00 2023-11-16T23:55:12.453 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-11-16T23:55:12.528 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-11-16T23:55:12.553 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2023-11-16T23:55:12.637 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-16T23:55:12.641 INFO:teuthology.orchestra.run.smithi006.stdout:(10/131): ceph-common-18.0.0-3906.g1c956514.el8 3.2 MB/s | 22 MB 00:06 2023-11-16T23:55:12.662 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-11-16T23:55:12.774 INFO:teuthology.orchestra.run.smithi006.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.9 MB/s | 258 kB 00:00 2023-11-16T23:55:12.849 INFO:teuthology.orchestra.run.smithi006.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 414 kB/s | 31 kB 00:00 2023-11-16T23:55:12.963 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 103 kB/s | 34 kB 00:00 2023-11-16T23:55:12.996 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 589 kB/s | 196 kB 00:00 2023-11-16T23:55:13.072 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2023-11-16T23:55:13.083 INFO:teuthology.orchestra.run.smithi006.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 3.2 MB/s | 771 kB 00:00 2023-11-16T23:55:13.097 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-11-16T23:55:13.167 INFO:teuthology.orchestra.run.smithi006.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.6 MB/s | 137 kB 00:00 2023-11-16T23:55:13.239 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-11-16T23:55:13.240 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-11-16T23:55:13.242 INFO:teuthology.orchestra.run.smithi006.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 1.8 MB/s | 139 kB 00:00 2023-11-16T23:55:13.330 INFO:teuthology.orchestra.run.smithi087.stdout:(25/131): librgw2-18.0.0-3906.g1c956514.el8.x86 2.8 MB/s | 6.5 MB 00:02 2023-11-16T23:55:13.334 INFO:teuthology.orchestra.run.smithi006.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 5.1 MB/s | 473 kB 00:00 2023-11-16T23:55:13.413 INFO:teuthology.orchestra.run.smithi087.stdout:(26/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 2.7 MB/s | 3.4 MB 00:01 2023-11-16T23:55:13.430 INFO:teuthology.orchestra.run.smithi087.stdout:(27/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-16T23:55:13.472 INFO:teuthology.orchestra.run.smithi087.stdout:(28/131): ceph-grafana-dashboards-18.0.0-3906.g 394 kB/s | 23 kB 00:00 2023-11-16T23:55:13.506 INFO:teuthology.orchestra.run.smithi087.stdout:(29/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.7 MB/s | 127 kB 00:00 2023-11-16T23:55:13.623 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): ceph-test-18.0.0-3906.g1c956514.el8.x 5.7 MB/s | 41 MB 00:07 2023-11-16T23:55:13.640 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 61 kB/s | 27 kB 00:00 2023-11-16T23:55:13.673 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 557 kB/s | 255 kB 00:00 2023-11-16T23:55:13.732 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 667 kB/s | 61 kB 00:00 2023-11-16T23:55:13.774 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2023-11-16T23:55:13.824 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 249 kB/s | 23 kB 00:00 2023-11-16T23:55:13.873 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-16T23:55:14.083 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 5.6 MB/s | 1.7 MB 00:00 2023-11-16T23:55:14.240 INFO:teuthology.orchestra.run.smithi087.stdout:(30/131): ceph-test-18.0.0-3906.g1c956514.el8.x 5.6 MB/s | 41 MB 00:07 2023-11-16T23:55:14.267 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 35 MB/s | 14 MB 00:00 2023-11-16T23:55:14.281 INFO:teuthology.orchestra.run.smithi087.stdout:(31/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 2.1 MB/s | 1.7 MB 00:00 2023-11-16T23:55:14.291 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): python3-webob-1.8.5-1.el8.1.noarch.rp 376 kB/s | 251 kB 00:00 2023-11-16T23:55:14.315 INFO:teuthology.orchestra.run.smithi087.stdout:(32/131): ceph-mgr-modules-core-18.0.0-3906.g1c 3.1 MB/s | 240 kB 00:00 2023-11-16T23:55:14.317 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): python3-pyasn1-modules-0.3.7-6.el8.no 471 kB/s | 110 kB 00:00 2023-11-16T23:55:14.340 INFO:teuthology.orchestra.run.smithi087.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 810 kB/s | 47 kB 00:00 2023-11-16T23:55:14.350 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 648 kB/s | 54 kB 00:00 2023-11-16T23:55:14.366 INFO:teuthology.orchestra.run.smithi087.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 291 kB/s | 15 kB 00:00 2023-11-16T23:55:14.367 INFO:teuthology.orchestra.run.smithi006.stdout:(17/131): librgw2-18.0.0-3906.g1c956514.el8.x86 6.3 MB/s | 6.5 MB 00:01 2023-11-16T23:55:14.384 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-11-16T23:55:14.391 INFO:teuthology.orchestra.run.smithi087.stdout:(35/131): ceph-volume-18.0.0-3906.g1c956514.el8 5.2 MB/s | 265 kB 00:00 2023-11-16T23:55:14.409 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 735 kB/s | 39 kB 00:00 2023-11-16T23:55:14.416 INFO:teuthology.orchestra.run.smithi087.stdout:(36/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.6 MB/s | 83 kB 00:00 2023-11-16T23:55:14.418 INFO:teuthology.orchestra.run.smithi006.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 898 kB/s | 45 kB 00:00 2023-11-16T23:55:14.435 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2023-11-16T23:55:14.442 INFO:teuthology.orchestra.run.smithi087.stdout:(37/131): cephfs-top-18.0.0-3906.g1c956514.el8. 358 kB/s | 18 kB 00:00 2023-11-16T23:55:14.468 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 8.0 MB/s | 688 kB 00:00 2023-11-16T23:55:14.476 INFO:teuthology.orchestra.run.smithi006.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 2.0 MB/s | 119 kB 00:00 2023-11-16T23:55:14.519 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-11-16T23:55:14.535 INFO:teuthology.orchestra.run.smithi006.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 3.3 MB/s | 197 kB 00:00 2023-11-16T23:55:14.560 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-11-16T23:55:14.577 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-11-16T23:55:14.611 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 7.5 MB/s | 713 kB 00:00 2023-11-16T23:55:14.618 INFO:teuthology.orchestra.run.smithi006.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 4.4 MB/s | 371 kB 00:00 2023-11-16T23:55:14.633 INFO:teuthology.orchestra.run.smithi087.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 307 kB/s | 59 kB 00:00 2023-11-16T23:55:14.636 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 454 kB/s | 27 kB 00:00 2023-11-16T23:55:14.670 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-11-16T23:55:14.675 INFO:teuthology.orchestra.run.smithi087.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-16T23:55:14.692 INFO:teuthology.orchestra.run.smithi087.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-11-16T23:55:14.695 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 751 kB/s | 44 kB 00:00 2023-11-16T23:55:14.702 INFO:teuthology.orchestra.run.smithi006.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 4.3 MB/s | 364 kB 00:00 2023-11-16T23:55:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-11-16T23:55:14.746 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-11-16T23:55:14.761 INFO:teuthology.orchestra.run.smithi006.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.8 MB/s | 107 kB 00:00 2023-11-16T23:55:14.768 INFO:teuthology.orchestra.run.smithi087.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.0 MB/s | 476 kB 00:00 2023-11-16T23:55:14.787 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-16T23:55:14.793 INFO:teuthology.orchestra.run.smithi087.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-11-16T23:55:14.811 INFO:teuthology.orchestra.run.smithi087.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 789 kB/s | 33 kB 00:00 2023-11-16T23:55:14.820 INFO:teuthology.orchestra.run.smithi006.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-16T23:55:14.836 INFO:teuthology.orchestra.run.smithi087.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 388 kB/s | 16 kB 00:00 2023-11-16T23:55:14.861 INFO:teuthology.orchestra.run.smithi087.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.3 MB/s | 173 kB 00:00 2023-11-16T23:55:14.888 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-11-16T23:55:14.895 INFO:teuthology.orchestra.run.smithi087.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-11-16T23:55:14.912 INFO:teuthology.orchestra.run.smithi087.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-16T23:55:14.913 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 511 kB/s | 64 kB 00:00 2023-11-16T23:55:14.937 INFO:teuthology.orchestra.run.smithi087.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2023-11-16T23:55:14.955 INFO:teuthology.orchestra.run.smithi087.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 419 kB/s | 18 kB 00:00 2023-11-16T23:55:14.980 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-11-16T23:55:14.988 INFO:teuthology.orchestra.run.smithi087.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-11-16T23:55:15.005 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 218 kB/s | 20 kB 00:00 2023-11-16T23:55:15.021 INFO:teuthology.orchestra.run.smithi087.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-11-16T23:55:15.022 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 944 kB/s | 126 kB 00:00 2023-11-16T23:55:15.047 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-11-16T23:55:15.063 INFO:teuthology.orchestra.run.smithi087.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-11-16T23:55:15.082 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-11-16T23:55:15.088 INFO:teuthology.orchestra.run.smithi087.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 901 kB/s | 60 kB 00:00 2023-11-16T23:55:15.105 INFO:teuthology.orchestra.run.smithi087.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2023-11-16T23:55:15.114 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-11-16T23:55:15.122 INFO:teuthology.orchestra.run.smithi087.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 465 kB/s | 16 kB 00:00 2023-11-16T23:55:15.148 INFO:teuthology.orchestra.run.smithi087.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 802 kB/s | 34 kB 00:00 2023-11-16T23:55:15.173 INFO:teuthology.orchestra.run.smithi087.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-16T23:55:15.181 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-11-16T23:55:15.198 INFO:teuthology.orchestra.run.smithi087.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-11-16T23:55:15.215 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 10 MB/s | 354 kB 00:00 2023-11-16T23:55:15.224 INFO:teuthology.orchestra.run.smithi087.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-11-16T23:55:15.249 INFO:teuthology.orchestra.run.smithi087.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-11-16T23:55:15.279 INFO:teuthology.orchestra.run.smithi006.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 7.5 MB/s | 3.4 MB 00:00 2023-11-16T23:55:15.282 INFO:teuthology.orchestra.run.smithi087.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 501 kB/s | 29 kB 00:00 2023-11-16T23:55:15.299 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-11-16T23:55:15.308 INFO:teuthology.orchestra.run.smithi087.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 470 kB/s | 27 kB 00:00 2023-11-16T23:55:15.337 INFO:teuthology.orchestra.run.smithi006.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 180 kB 00:00 2023-11-16T23:55:15.341 INFO:teuthology.orchestra.run.smithi087.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2023-11-16T23:55:15.366 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-16T23:55:15.367 INFO:teuthology.orchestra.run.smithi087.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-11-16T23:55:15.379 INFO:teuthology.orchestra.run.smithi006.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 555 kB/s | 23 kB 00:00 2023-11-16T23:55:15.392 INFO:teuthology.orchestra.run.smithi087.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-16T23:55:15.408 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-11-16T23:55:15.410 INFO:teuthology.orchestra.run.smithi087.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-11-16T23:55:15.430 INFO:teuthology.orchestra.run.smithi006.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 2.5 MB/s | 127 kB 00:00 2023-11-16T23:55:15.434 INFO:teuthology.orchestra.run.smithi087.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 550 kB/s | 23 kB 00:00 2023-11-16T23:55:15.460 INFO:teuthology.orchestra.run.smithi087.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-16T23:55:15.489 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-16T23:55:15.517 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 432 kB/s | 47 kB 00:00 2023-11-16T23:55:15.527 INFO:teuthology.orchestra.run.smithi087.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-11-16T23:55:15.541 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.7 MB/s | 200 kB 00:00 2023-11-16T23:55:15.559 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-11-16T23:55:15.619 INFO:teuthology.orchestra.run.smithi087.stdout:(71/131): ceph-mgr-diskprediction-local-18.0.0- 3.5 MB/s | 7.4 MB 00:02 2023-11-16T23:55:15.626 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-11-16T23:55:15.643 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 575 kB/s | 48 kB 00:00 2023-11-16T23:55:15.644 INFO:teuthology.orchestra.run.smithi087.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 254 kB/s | 47 kB 00:00 2023-11-16T23:55:15.646 INFO:teuthology.orchestra.run.smithi006.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 7.9 MB/s | 1.7 MB 00:00 2023-11-16T23:55:15.668 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-16T23:55:15.677 INFO:teuthology.orchestra.run.smithi087.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2023-11-16T23:55:15.693 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-11-16T23:55:15.694 INFO:teuthology.orchestra.run.smithi087.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 609 kB/s | 43 kB 00:00 2023-11-16T23:55:15.718 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 566 kB/s | 43 kB 00:00 2023-11-16T23:55:15.720 INFO:teuthology.orchestra.run.smithi087.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-11-16T23:55:15.737 INFO:teuthology.orchestra.run.smithi087.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-11-16T23:55:15.744 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2023-11-16T23:55:15.762 INFO:teuthology.orchestra.run.smithi087.stdout:(77/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.9 MB/s | 220 kB 00:00 2023-11-16T23:55:15.769 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-11-16T23:55:15.787 INFO:teuthology.orchestra.run.smithi087.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 637 kB/s | 43 kB 00:00 2023-11-16T23:55:15.813 INFO:teuthology.orchestra.run.smithi087.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-11-16T23:55:15.826 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 229 kB/s | 19 kB 00:00 2023-11-16T23:55:15.838 INFO:teuthology.orchestra.run.smithi087.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-11-16T23:55:15.861 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.6 MB/s | 155 kB 00:00 2023-11-16T23:55:15.863 INFO:teuthology.orchestra.run.smithi087.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-11-16T23:55:15.880 INFO:teuthology.orchestra.run.smithi087.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-11-16T23:55:15.895 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-11-16T23:55:15.905 INFO:teuthology.orchestra.run.smithi087.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.6 MB/s | 560 kB 00:00 2023-11-16T23:55:15.920 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-11-16T23:55:15.930 INFO:teuthology.orchestra.run.smithi087.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 358 kB/s | 18 kB 00:00 2023-11-16T23:55:15.964 INFO:teuthology.orchestra.run.smithi087.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.4 MB/s | 559 kB 00:00 2023-11-16T23:55:15.979 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-11-16T23:55:15.990 INFO:teuthology.orchestra.run.smithi087.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-16T23:55:16.005 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-11-16T23:55:16.015 INFO:teuthology.orchestra.run.smithi087.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-11-16T23:55:16.029 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-11-16T23:55:16.040 INFO:teuthology.orchestra.run.smithi087.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-11-16T23:55:16.055 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-11-16T23:55:16.065 INFO:teuthology.orchestra.run.smithi087.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-16T23:55:16.080 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-16T23:55:16.091 INFO:teuthology.orchestra.run.smithi087.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-16T23:55:16.105 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-16T23:55:16.107 INFO:teuthology.orchestra.run.smithi087.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-16T23:55:16.141 INFO:teuthology.orchestra.run.smithi087.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-16T23:55:16.164 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.1 MB/s | 98 kB 00:00 2023-11-16T23:55:16.167 INFO:teuthology.orchestra.run.smithi087.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 660 kB/s | 39 kB 00:00 2023-11-16T23:55:16.189 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 691 kB/s | 58 kB 00:00 2023-11-16T23:55:16.200 INFO:teuthology.orchestra.run.smithi087.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-16T23:55:16.214 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 772 kB/s | 39 kB 00:00 2023-11-16T23:55:16.225 INFO:teuthology.orchestra.run.smithi087.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-11-16T23:55:16.242 INFO:teuthology.orchestra.run.smithi087.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-11-16T23:55:16.256 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.5 MB/s | 644 kB 00:00 2023-11-16T23:55:16.281 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-11-16T23:55:16.293 INFO:teuthology.orchestra.run.smithi087.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2023-11-16T23:55:16.307 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-16T23:55:16.339 INFO:teuthology.orchestra.run.smithi006.stdout:(30/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 2.5 MB/s | 18 MB 00:07 2023-11-16T23:55:16.339 INFO:teuthology.orchestra.run.smithi087.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-11-16T23:55:16.365 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-11-16T23:55:16.368 INFO:teuthology.orchestra.run.smithi087.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 717 kB/s | 54 kB 00:00 2023-11-16T23:55:16.402 INFO:teuthology.orchestra.run.smithi087.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.1 MB/s | 259 kB 00:00 2023-11-16T23:55:16.432 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-11-16T23:55:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-11-16T23:55:16.461 INFO:teuthology.orchestra.run.smithi087.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-11-16T23:55:16.481 INFO:teuthology.orchestra.run.smithi006.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 1.7 MB/s | 240 kB 00:00 2023-11-16T23:55:16.503 INFO:teuthology.orchestra.run.smithi087.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-11-16T23:55:16.536 INFO:teuthology.orchestra.run.smithi087.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-11-16T23:55:16.562 INFO:teuthology.orchestra.run.smithi087.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-11-16T23:55:16.587 INFO:teuthology.orchestra.run.smithi087.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-11-16T23:55:16.605 INFO:teuthology.orchestra.run.smithi006.stdout:(32/131): ceph-mgr-diskprediction-local-18.0.0- 7.7 MB/s | 7.4 MB 00:00 2023-11-16T23:55:16.629 INFO:teuthology.orchestra.run.smithi087.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-11-16T23:55:16.631 INFO:teuthology.orchestra.run.smithi006.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 316 kB/s | 47 kB 00:00 2023-11-16T23:55:16.654 INFO:teuthology.orchestra.run.smithi087.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 1.1 MB/s | 27 kB 00:00 2023-11-16T23:55:16.664 INFO:teuthology.orchestra.run.smithi006.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 248 kB/s | 15 kB 00:00 2023-11-16T23:55:16.696 INFO:teuthology.orchestra.run.smithi087.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-16T23:55:16.721 INFO:teuthology.orchestra.run.smithi087.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-11-16T23:55:16.731 INFO:teuthology.orchestra.run.smithi006.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.2 MB/s | 83 kB 00:00 2023-11-16T23:55:16.813 INFO:teuthology.orchestra.run.smithi087.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-11-16T23:55:16.815 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 6.8 MB/s | 3.7 MB 00:00 2023-11-16T23:55:16.864 INFO:teuthology.orchestra.run.smithi087.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-11-16T23:55:16.906 INFO:teuthology.orchestra.run.smithi087.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-16T23:55:16.957 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 5.5 MB/s | 3.6 MB 00:00 2023-11-16T23:55:16.962 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:55:16.963 INFO:teuthology.orchestra.run.smithi186.stdout:Total 13 MB/s | 197 MB 00:14 2023-11-16T23:55:16.964 INFO:teuthology.orchestra.run.smithi087.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-11-16T23:55:16.990 INFO:teuthology.orchestra.run.smithi087.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 767 kB/s | 64 kB 00:00 2023-11-16T23:55:17.015 INFO:teuthology.orchestra.run.smithi087.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-16T23:55:17.040 INFO:teuthology.orchestra.run.smithi087.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-11-16T23:55:17.066 INFO:teuthology.orchestra.run.smithi087.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-16T23:55:17.108 INFO:teuthology.orchestra.run.smithi087.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-16T23:55:17.133 INFO:teuthology.orchestra.run.smithi006.stdout:(36/131): ceph-test-18.0.0-3906.g1c956514.el8.x 8.5 MB/s | 41 MB 00:04 2023-11-16T23:55:17.141 INFO:teuthology.orchestra.run.smithi087.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 5.8 MB/s | 198 kB 00:00 2023-11-16T23:55:17.157 INFO:teuthology.orchestra.run.smithi006.stdout:(37/131): cephfs-top-18.0.0-3906.g1c956514.el8. 42 kB/s | 18 kB 00:00 2023-11-16T23:55:17.182 INFO:teuthology.orchestra.run.smithi006.stdout:(38/131): ceph-volume-18.0.0-3906.g1c956514.el8 480 kB/s | 265 kB 00:00 2023-11-16T23:55:17.291 INFO:teuthology.orchestra.run.smithi006.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 440 kB/s | 59 kB 00:00 2023-11-16T23:55:17.300 INFO:teuthology.orchestra.run.smithi087.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-11-16T23:55:17.317 INFO:teuthology.orchestra.run.smithi006.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-11-16T23:55:17.342 INFO:teuthology.orchestra.run.smithi087.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-16T23:55:17.367 INFO:teuthology.orchestra.run.smithi006.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 881 kB/s | 44 kB 00:00 2023-11-16T23:55:17.384 INFO:teuthology.orchestra.run.smithi087.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-16T23:55:17.392 INFO:teuthology.orchestra.run.smithi006.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-16T23:55:17.426 INFO:teuthology.orchestra.run.smithi006.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-11-16T23:55:17.450 INFO:teuthology.orchestra.run.smithi087.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-11-16T23:55:17.451 INFO:teuthology.orchestra.run.smithi006.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 564 kB/s | 33 kB 00:00 2023-11-16T23:55:17.476 INFO:teuthology.orchestra.run.smithi006.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-11-16T23:55:17.501 INFO:teuthology.orchestra.run.smithi087.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-11-16T23:55:17.501 INFO:teuthology.orchestra.run.smithi006.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-11-16T23:55:17.529 INFO:teuthology.orchestra.run.smithi006.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-16T23:55:17.552 INFO:teuthology.orchestra.run.smithi006.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-11-16T23:55:17.559 INFO:teuthology.orchestra.run.smithi087.stdout:(126/131): openblas-0.3.15-3.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-11-16T23:55:17.569 INFO:teuthology.orchestra.run.smithi006.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-11-16T23:55:17.594 INFO:teuthology.orchestra.run.smithi006.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 877 kB/s | 81 kB 00:00 2023-11-16T23:55:17.610 INFO:teuthology.orchestra.run.smithi087.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-11-16T23:55:17.621 INFO:teuthology.orchestra.run.smithi006.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-11-16T23:55:17.637 INFO:teuthology.orchestra.run.smithi006.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-11-16T23:55:17.654 INFO:teuthology.orchestra.run.smithi006.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-11-16T23:55:17.671 INFO:teuthology.orchestra.run.smithi006.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-11-16T23:55:17.694 INFO:teuthology.orchestra.run.smithi087.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-11-16T23:55:17.744 INFO:teuthology.orchestra.run.smithi087.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-16T23:55:17.746 INFO:teuthology.orchestra.run.smithi006.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-11-16T23:55:17.763 INFO:teuthology.orchestra.run.smithi006.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-11-16T23:55:17.792 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-11-16T23:55:17.805 INFO:teuthology.orchestra.run.smithi006.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 4.9 MB/s | 1.2 MB 00:00 2023-11-16T23:55:17.822 INFO:teuthology.orchestra.run.smithi006.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 448 kB/s | 34 kB 00:00 2023-11-16T23:55:17.855 INFO:teuthology.orchestra.run.smithi006.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-11-16T23:55:17.881 INFO:teuthology.orchestra.run.smithi006.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 856 kB/s | 64 kB 00:00 2023-11-16T23:55:17.898 INFO:teuthology.orchestra.run.smithi006.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-11-16T23:55:17.913 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-11-16T23:55:17.913 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-11-16T23:55:17.923 INFO:teuthology.orchestra.run.smithi006.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 356 kB/s | 24 kB 00:00 2023-11-16T23:55:17.948 INFO:teuthology.orchestra.run.smithi006.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 585 kB/s | 29 kB 00:00 2023-11-16T23:55:17.974 INFO:teuthology.orchestra.run.smithi006.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-11-16T23:55:17.999 INFO:teuthology.orchestra.run.smithi006.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-11-16T23:55:18.024 INFO:teuthology.orchestra.run.smithi006.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-16T23:55:18.035 INFO:teuthology.orchestra.run.smithi087.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 7.7 MB/s | 3.7 MB 00:00 2023-11-16T23:55:18.049 INFO:teuthology.orchestra.run.smithi006.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-16T23:55:18.074 INFO:teuthology.orchestra.run.smithi006.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 460 kB/s | 23 kB 00:00 2023-11-16T23:55:18.100 INFO:teuthology.orchestra.run.smithi006.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-11-16T23:55:18.129 INFO:teuthology.orchestra.run.smithi006.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-16T23:55:18.175 INFO:teuthology.orchestra.run.smithi006.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 624 kB/s | 47 kB 00:00 2023-11-16T23:55:18.217 INFO:teuthology.orchestra.run.smithi006.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-11-16T23:55:18.243 INFO:teuthology.orchestra.run.smithi006.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 640 kB/s | 43 kB 00:00 2023-11-16T23:55:18.276 INFO:teuthology.orchestra.run.smithi006.stdout:(74/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-11-16T23:55:18.301 INFO:teuthology.orchestra.run.smithi006.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 962 kB/s | 48 kB 00:00 2023-11-16T23:55:18.319 INFO:teuthology.orchestra.run.smithi087.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 5.0 MB/s | 3.6 MB 00:00 2023-11-16T23:55:18.322 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:55:18.322 INFO:teuthology.orchestra.run.smithi087.stdout:Total 12 MB/s | 197 MB 00:16 2023-11-16T23:55:18.327 INFO:teuthology.orchestra.run.smithi006.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-16T23:55:18.383 INFO:teuthology.orchestra.run.smithi006.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 529 kB/s | 43 kB 00:00 2023-11-16T23:55:18.410 INFO:teuthology.orchestra.run.smithi006.stdout:(78/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-11-16T23:55:18.439 INFO:teuthology.orchestra.run.smithi006.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-11-16T23:55:18.461 INFO:teuthology.orchestra.run.smithi006.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.9 MB/s | 229 kB 00:00 2023-11-16T23:55:18.487 INFO:teuthology.orchestra.run.smithi006.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 370 kB/s | 19 kB 00:00 2023-11-16T23:55:18.512 INFO:teuthology.orchestra.run.smithi006.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-16T23:55:18.546 INFO:teuthology.orchestra.run.smithi006.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-16T23:55:18.571 INFO:teuthology.orchestra.run.smithi006.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-11-16T23:55:18.613 INFO:teuthology.orchestra.run.smithi006.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-11-16T23:55:18.646 INFO:teuthology.orchestra.run.smithi006.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-11-16T23:55:18.672 INFO:teuthology.orchestra.run.smithi006.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-16T23:55:18.697 INFO:teuthology.orchestra.run.smithi006.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-11-16T23:55:18.723 INFO:teuthology.orchestra.run.smithi006.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-16T23:55:18.753 INFO:teuthology.orchestra.run.smithi006.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-16T23:55:18.773 INFO:teuthology.orchestra.run.smithi006.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-11-16T23:55:18.798 INFO:teuthology.orchestra.run.smithi006.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-16T23:55:18.815 INFO:teuthology.orchestra.run.smithi006.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 921 kB/s | 39 kB 00:00 2023-11-16T23:55:18.858 INFO:teuthology.orchestra.run.smithi006.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-16T23:55:18.882 INFO:teuthology.orchestra.run.smithi006.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-11-16T23:55:18.908 INFO:teuthology.orchestra.run.smithi006.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-16T23:55:18.988 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-11-16T23:55:18.992 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-11-16T23:55:19.016 INFO:teuthology.orchestra.run.smithi006.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.0 MB/s | 110 kB 00:00 2023-11-16T23:55:19.075 INFO:teuthology.orchestra.run.smithi006.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2023-11-16T23:55:19.092 INFO:teuthology.orchestra.run.smithi006.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 721 kB/s | 54 kB 00:00 2023-11-16T23:55:19.125 INFO:teuthology.orchestra.run.smithi006.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-16T23:55:19.159 INFO:teuthology.orchestra.run.smithi006.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-11-16T23:55:19.174 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2023-11-16T23:55:19.184 INFO:teuthology.orchestra.run.smithi006.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-11-16T23:55:19.235 INFO:teuthology.orchestra.run.smithi006.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-11-16T23:55:19.279 INFO:teuthology.orchestra.run.smithi006.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.7 MB/s | 538 kB 00:00 2023-11-16T23:55:19.295 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2023-11-16T23:55:19.295 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2023-11-16T23:55:19.311 INFO:teuthology.orchestra.run.smithi006.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-11-16T23:55:19.344 INFO:teuthology.orchestra.run.smithi006.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-11-16T23:55:19.394 INFO:teuthology.orchestra.run.smithi006.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-11-16T23:55:19.428 INFO:teuthology.orchestra.run.smithi006.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 801 kB/s | 27 kB 00:00 2023-11-16T23:55:19.478 INFO:teuthology.orchestra.run.smithi006.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-11-16T23:55:19.520 INFO:teuthology.orchestra.run.smithi006.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-11-16T23:55:19.687 INFO:teuthology.orchestra.run.smithi006.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-11-16T23:55:19.737 INFO:teuthology.orchestra.run.smithi006.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-11-16T23:55:19.804 INFO:teuthology.orchestra.run.smithi006.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-11-16T23:55:19.838 INFO:teuthology.orchestra.run.smithi006.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-11-16T23:55:19.855 INFO:teuthology.orchestra.run.smithi006.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-11-16T23:55:19.889 INFO:teuthology.orchestra.run.smithi006.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-16T23:55:19.931 INFO:teuthology.orchestra.run.smithi006.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 266 kB/s | 20 kB 00:00 2023-11-16T23:55:19.956 INFO:teuthology.orchestra.run.smithi006.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.5 MB/s | 103 kB 00:00 2023-11-16T23:55:19.998 INFO:teuthology.orchestra.run.smithi006.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-11-16T23:55:20.040 INFO:teuthology.orchestra.run.smithi006.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-16T23:55:20.224 INFO:teuthology.orchestra.run.smithi006.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 10 MB/s | 14 MB 00:01 2023-11-16T23:55:20.292 INFO:teuthology.orchestra.run.smithi006.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-11-16T23:55:20.349 INFO:teuthology.orchestra.run.smithi006.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.1 MB/s | 354 kB 00:00 2023-11-16T23:55:20.370 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2023-11-16T23:55:20.374 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2023-11-16T23:55:20.433 INFO:teuthology.orchestra.run.smithi006.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 10 MB/s | 4.9 MB 00:00 2023-11-16T23:55:20.533 INFO:teuthology.orchestra.run.smithi006.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 9.6 MB/s | 4.7 MB 00:00 2023-11-16T23:55:20.567 INFO:teuthology.orchestra.run.smithi006.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-11-16T23:55:20.634 INFO:teuthology.orchestra.run.smithi006.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-16T23:55:20.726 INFO:teuthology.orchestra.run.smithi006.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-11-16T23:55:20.776 INFO:teuthology.orchestra.run.smithi006.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-16T23:55:21.151 INFO:teuthology.orchestra.run.smithi006.stdout:(130/131): librbd1-18.0.0-3906.g1c956514.el8.x8 6.9 MB/s | 3.6 MB 00:00 2023-11-16T23:55:21.573 INFO:teuthology.orchestra.run.smithi006.stdout:(131/131): librados2-18.0.0-3906.g1c956514.el8. 3.6 MB/s | 3.7 MB 00:01 2023-11-16T23:55:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:55:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:Total 12 MB/s | 197 MB 00:15 2023-11-16T23:55:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-11-16T23:55:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-11-16T23:55:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-11-16T23:55:24.073 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-11-16T23:55:24.270 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T23:55:24.425 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T23:55:24.653 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T23:55:24.805 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T23:55:24.920 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T23:55:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-11-16T23:55:25.031 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-11-16T23:55:25.155 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T23:55:25.197 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2023-11-16T23:55:25.345 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T23:55:25.562 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T23:55:25.583 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T23:55:25.722 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-16T23:55:25.796 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:25.822 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-16T23:55:26.055 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-16T23:55:26.217 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:26.413 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:26.456 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-16T23:55:26.567 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:26.647 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-16T23:55:26.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:55:26.863 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:55:27.026 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-16T23:55:27.039 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-16T23:55:27.230 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:27.284 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T23:55:27.352 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:27.508 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T23:55:27.522 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:27.615 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:27.768 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:55:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T23:55:27.938 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-16T23:55:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T23:55:28.184 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T23:55:28.272 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:28.376 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:28.408 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T23:55:28.546 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:28.639 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:28.787 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T23:55:28.808 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T23:55:28.931 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T23:55:29.041 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T23:55:29.172 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:29.271 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:29.396 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T23:55:29.430 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:29.573 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:29.817 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-16T23:55:29.978 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-16T23:55:30.021 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T23:55:30.204 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T23:55:30.303 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-11-16T23:55:30.487 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T23:55:30.724 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T23:55:30.737 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-16T23:55:30.793 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T23:55:30.880 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-16T23:55:30.896 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-16T23:55:30.971 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T23:55:31.010 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-16T23:55:31.045 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-16T23:55:31.171 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-16T23:55:31.530 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-16T23:55:31.692 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T23:55:31.789 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-16T23:55:31.946 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-16T23:55:31.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T23:55:32.005 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-16T23:55:32.198 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T23:55:32.221 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-16T23:55:32.449 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T23:55:32.545 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-16T23:55:32.668 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T23:55:32.699 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-16T23:55:32.721 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-16T23:55:32.851 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T23:55:32.888 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:33.024 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:55:33.113 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T23:55:33.254 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:33.319 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-16T23:55:33.374 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:55:33.421 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-16T23:55:33.457 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-16T23:55:33.552 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:55:33.654 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-16T23:55:33.743 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-16T23:55:33.852 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-16T23:55:34.114 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T23:55:34.186 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T23:55:34.278 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-16T23:55:34.390 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-16T23:55:34.417 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-16T23:55:34.453 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T23:55:34.642 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-16T23:55:34.770 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T23:55:34.863 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-16T23:55:34.909 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T23:55:35.081 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T23:55:35.182 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T23:55:35.338 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:35.361 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T23:55:35.466 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-16T23:55:35.529 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T23:55:35.621 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:35.824 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-16T23:55:36.086 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-16T23:55:36.281 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-16T23:55:36.589 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-16T23:55:36.631 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-16T23:55:36.721 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-16T23:55:36.895 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-16T23:55:37.096 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-16T23:55:37.295 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T23:55:37.625 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-16T23:55:37.638 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T23:55:37.811 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-16T23:55:37.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T23:55:37.943 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-16T23:55:37.996 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-16T23:55:38.047 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T23:55:38.223 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-16T23:55:38.250 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-16T23:55:38.424 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-16T23:55:38.449 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-16T23:55:38.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T23:55:38.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T23:55:38.699 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-16T23:55:38.779 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T23:55:38.849 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T23:55:38.920 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-16T23:55:38.989 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T23:55:39.018 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-16T23:55:39.106 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T23:55:39.168 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-16T23:55:39.319 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-16T23:55:39.350 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T23:55:39.463 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-16T23:55:39.570 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T23:55:39.603 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-16T23:55:39.672 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-16T23:55:39.804 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-16T23:55:39.910 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-16T23:55:39.920 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T23:55:40.053 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-16T23:55:40.115 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-16T23:55:40.256 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T23:55:40.270 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-16T23:55:40.317 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-16T23:55:40.350 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-16T23:55:40.489 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T23:55:40.576 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T23:55:40.591 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-16T23:55:40.697 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T23:55:40.720 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-16T23:55:40.757 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-16T23:55:40.876 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-16T23:55:40.920 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-16T23:55:40.966 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T23:55:41.081 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-16T23:55:41.138 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-16T23:55:41.231 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-16T23:55:41.320 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-16T23:55:41.320 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T23:55:41.379 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-16T23:55:41.552 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T23:55:41.570 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-16T23:55:41.746 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T23:55:41.777 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-16T23:55:41.991 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T23:55:42.152 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-16T23:55:42.190 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T23:55:42.356 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T23:55:42.480 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-16T23:55:42.683 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T23:55:42.698 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-16T23:55:42.915 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-16T23:55:42.959 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T23:55:43.149 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-16T23:55:43.233 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T23:55:43.444 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-16T23:55:43.459 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T23:55:43.479 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-16T23:55:43.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T23:55:43.644 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-16T23:55:43.795 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T23:55:43.838 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-16T23:55:44.002 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T23:55:44.055 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T23:55:44.082 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-16T23:55:44.258 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-16T23:55:44.322 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T23:55:44.406 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-16T23:55:44.555 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T23:55:44.644 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-16T23:55:44.714 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-16T23:55:44.752 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T23:55:44.816 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T23:55:44.966 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-16T23:55:45.014 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-16T23:55:45.041 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T23:55:45.191 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-16T23:55:45.219 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T23:55:45.288 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-16T23:55:45.393 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T23:55:45.429 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-16T23:55:45.546 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T23:55:45.573 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-16T23:55:45.611 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T23:55:45.797 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T23:55:45.811 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T23:55:45.825 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T23:55:46.037 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T23:55:46.069 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T23:55:46.083 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T23:55:46.223 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T23:55:46.262 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-16T23:55:46.267 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T23:55:46.437 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T23:55:46.448 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T23:55:46.571 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T23:55:46.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-16T23:55:46.626 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T23:55:46.752 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T23:55:46.845 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T23:55:46.964 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T23:55:47.062 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T23:55:47.064 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-16T23:55:47.141 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T23:55:47.238 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T23:55:47.268 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-16T23:55:47.330 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T23:55:47.424 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T23:55:47.453 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-16T23:55:47.467 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T23:55:47.548 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-16T23:55:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T23:55:47.691 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T23:55:47.813 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-16T23:55:47.862 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T23:55:47.923 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T23:55:48.057 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T23:55:48.060 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-16T23:55:48.142 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T23:55:48.228 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-16T23:55:48.267 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T23:55:48.335 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-16T23:55:48.391 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T23:55:48.508 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T23:55:48.600 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-16T23:55:48.610 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T23:55:48.716 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T23:55:48.792 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-16T23:55:48.793 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T23:55:48.930 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T23:55:48.964 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-16T23:55:49.037 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T23:55:49.163 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-16T23:55:49.364 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-16T23:55:49.535 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-16T23:55:49.779 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-16T23:55:50.023 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-16T23:55:50.050 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T23:55:50.113 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T23:55:50.373 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T23:55:50.396 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T23:55:50.555 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-16T23:55:50.617 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T23:55:50.651 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T23:55:50.894 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-16T23:55:50.925 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T23:55:50.969 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T23:55:51.153 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-16T23:55:51.202 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T23:55:51.257 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T23:55:51.412 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T23:55:51.414 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-16T23:55:51.483 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T23:55:51.673 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-16T23:55:52.079 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-16T23:55:52.183 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T23:55:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T23:55:52.304 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-16T23:55:52.377 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:55:52.384 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:55:52.444 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:55:52.449 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:55:52.538 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-16T23:55:52.602 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-16T23:55:52.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-16T23:55:52.773 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-16T23:55:52.953 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-16T23:55:53.142 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-16T23:55:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-16T23:55:53.855 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-16T23:55:54.129 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-16T23:55:54.398 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-16T23:55:54.563 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T23:55:54.753 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T23:55:55.067 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T23:55:55.286 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T23:55:55.432 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T23:55:55.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T23:55:55.836 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T23:55:56.062 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T23:55:56.188 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:56.302 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T23:55:56.434 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:56.506 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T23:55:56.652 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:56.668 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T23:55:56.684 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:56.932 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T23:55:57.159 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T23:55:57.226 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:57.259 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:55:57.360 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T23:55:57.513 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:55:57.529 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:55:57.592 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:55:57.638 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:55:57.683 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:55:57.767 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:55:57.936 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T23:55:58.147 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T23:55:58.392 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T23:55:59.787 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T23:56:00.099 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T23:56:00.368 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T23:56:00.751 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T23:56:01.032 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T23:56:01.233 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T23:56:02.109 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T23:56:02.359 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:56:02.440 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-16T23:56:02.635 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-16T23:56:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:56:07.960 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:56:08.701 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-16T23:56:09.045 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:56:09.187 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-16T23:56:09.342 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:56:16.365 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:56:16.365 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2023-11-16T23:56:16.365 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2023-11-16T23:56:16.365 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /dev 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /run 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /dev 2023-11-16T23:56:16.366 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:56:16.708 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:16.799 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-11-16T23:56:16.800 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:56:17.148 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:18.270 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:18.645 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:18.717 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:18.841 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:18.901 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:19.120 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:19.290 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:19.323 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:19.465 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:19.483 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:19.775 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:19.981 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:19.998 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:21.104 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:21.338 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:21.463 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-16T23:56:21.672 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-16T23:56:21.706 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:21.856 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:21.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-16T23:56:22.072 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:22.294 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-16T23:56:22.359 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:22.496 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:22.712 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:23.016 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:23.322 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:23.473 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:23.566 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:23.945 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-16T23:56:23.983 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:24.146 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:24.172 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:24.286 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:24.287 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:24.323 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:24.334 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:24.645 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-16T23:56:24.880 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:24.974 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:25.571 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-16T23:56:25.752 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:25.788 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:25.923 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:25.966 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:26.089 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:26.329 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-16T23:56:26.433 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:26.525 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:26.559 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:26.581 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:26.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:26.779 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:26.938 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:27.283 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:27.471 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:30.480 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:30.518 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-16T23:56:30.518 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /run 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-11-16T23:56:30.519 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:56:30.924 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:31.369 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-16T23:56:31.382 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:31.669 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-16T23:56:31.687 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-16T23:56:31.833 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 117/134 2023-11-16T23:56:31.837 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T23:56:32.027 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T23:56:32.139 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-16T23:56:32.296 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-16T23:56:32.485 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-16T23:56:32.692 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-16T23:56:32.750 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-16T23:56:32.909 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-16T23:56:32.933 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:32.937 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T23:56:33.039 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T23:56:33.215 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-16T23:56:33.268 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T23:56:33.389 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:33.423 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-16T23:56:33.632 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T23:56:33.841 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-16T23:56:33.950 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-16T23:56:33.974 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T23:56:34.164 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:34.207 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-16T23:56:35.197 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:35.463 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:35.483 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:35.678 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-16T23:56:35.945 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-16T23:56:36.250 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-16T23:56:36.264 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:36.528 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:36.545 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:56:36.614 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T23:56:36.706 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-16T23:56:36.804 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-16T23:56:36.898 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-16T23:56:37.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-16T23:56:37.138 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-16T23:56:37.150 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-16T23:56:37.200 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:37.304 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-16T23:56:37.313 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-16T23:56:37.315 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-16T23:56:37.378 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-16T23:56:37.440 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-16T23:56:37.440 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:37.497 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-16T23:56:37.624 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:37.624 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:56:37.705 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T23:56:37.727 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-16T23:56:37.818 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:56:37.853 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T23:56:37.861 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T23:56:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:38.020 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:38.082 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T23:56:38.182 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-16T23:56:38.277 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T23:56:38.371 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T23:56:38.425 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-16T23:56:38.530 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-16T23:56:38.531 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-16T23:56:38.532 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-16T23:56:38.533 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-16T23:56:38.545 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T23:56:38.546 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T23:56:38.547 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T23:56:38.548 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T23:56:38.549 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T23:56:38.550 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-16T23:56:38.552 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-16T23:56:38.553 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-16T23:56:38.553 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-16T23:56:38.553 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-16T23:56:38.553 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-16T23:56:38.553 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-16T23:56:38.554 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-16T23:56:38.554 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-16T23:56:38.554 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-16T23:56:38.555 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-16T23:56:38.556 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-16T23:56:38.557 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-16T23:56:38.558 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-16T23:56:38.558 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-16T23:56:38.558 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:38.568 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T23:56:38.569 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T23:56:38.574 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T23:56:38.574 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:38.781 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:38.781 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:56:39.042 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:56:39.252 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:39.582 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-16T23:56:39.747 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-16T23:56:39.811 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-16T23:56:39.812 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-16T23:56:39.813 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T23:56:39.814 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T23:56:39.815 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T23:56:39.816 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T23:56:39.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T23:56:39.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T23:56:39.819 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T23:56:39.820 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T23:56:39.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:56:39.822 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T23:56:39.881 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-16T23:56:40.282 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-16T23:56:40.513 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:40.581 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-16T23:56:40.780 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:40.810 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-16T23:56:41.114 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:41.114 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:01:51 ago on Thu 16 Nov 2023 11:54:50 PM UTC. 2023-11-16T23:56:42.269 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-16T23:56:42.325 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:56:42.325 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:52 ago on Thu 16 Nov 2023 11:54:50 PM UTC. 2023-11-16T23:56:42.485 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:42.550 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-16T23:56:42.713 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:42.786 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-16T23:56:42.963 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:43.394 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-16T23:56:43.596 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:43.660 INFO:teuthology.orchestra.run.smithi087.stdout:Installed products updated. 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.874 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.875 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.876 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: jq-1.6-3.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T23:56:43.877 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T23:56:43.878 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:43.879 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T23:56:43.880 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T23:56:43.881 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T23:56:43.882 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T23:56:43.883 INFO:teuthology.orchestra.run.smithi087.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T23:56:43.884 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T23:56:43.885 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T23:56:43.885 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-11-16T23:56:43.885 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2023-11-16T23:56:44.082 DEBUG:teuthology.parallel:result is None 2023-11-16T23:56:44.766 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T23:56:44.982 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-11-16T23:56:44.984 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.985 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T23:56:44.986 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T23:56:44.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T23:56:44.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T23:56:44.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.991 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:56:44.992 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-11-16T23:56:45.165 DEBUG:teuthology.parallel:result is None 2023-11-16T23:56:48.132 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-16T23:56:49.542 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-16T23:56:49.979 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-16T23:56:50.177 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 117/134 2023-11-16T23:56:50.453 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-16T23:56:50.628 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-16T23:56:50.890 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-16T23:56:51.159 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-16T23:56:51.351 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-16T23:56:51.513 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T23:56:55.162 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:57:00.449 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-16T23:57:00.450 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-16T23:57:00.451 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-16T23:57:00.452 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-16T23:57:00.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T23:57:00.454 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T23:57:08.125 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T23:57:08.162 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T23:57:08.231 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T23:57:08.262 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T23:57:08.289 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T23:57:08.289 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T23:57:08.289 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T23:57:08.289 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T23:57:08.561 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T23:57:08.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T23:57:08.563 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T23:57:08.642 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-16T23:57:08.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-16T23:57:08.669 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-16T23:57:08.670 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-16T23:57:08.671 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-16T23:57:08.729 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T23:57:08.730 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T23:57:08.732 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T23:57:08.732 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:10 ago on Thu 16 Nov 2023 11:54:52 PM UTC. 2023-11-16T23:57:08.732 INFO:teuthology.orchestra.run.smithi006.stdout:Installed products updated. 2023-11-16T23:57:08.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:57:08.732 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.733 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.762 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.783 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T23:57:08.784 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-3.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T23:57:08.785 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T23:57:08.786 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T23:57:08.786 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.786 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T23:57:08.843 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:57:08.844 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T23:57:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T23:57:08.872 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T23:57:08.873 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T23:57:08.874 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T23:57:08.874 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T23:57:08.874 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T23:57:08.965 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T23:57:08.966 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T23:57:08.967 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T23:57:08.967 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T23:57:08.967 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T23:57:08.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:57:08.967 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-16T23:57:08.967 DEBUG:teuthology.parallel:result is None 2023-11-16T23:57:08.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:09.487 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T23:57:09.548 INFO:teuthology.orchestra.run.smithi006.stdout:18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.549 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.549 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-16T23:57:09.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:09.688 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T23:57:09.718 INFO:teuthology.orchestra.run.smithi087.stdout:18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.719 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.719 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-16T23:57:09.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:09.932 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T23:57:09.962 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.963 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-16T23:57:09.963 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-16T23:57:09.963 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-16T23:57:09.963 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:57:09.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T23:57:10.062 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:57:10.063 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T23:57:10.110 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:57:10.110 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T23:57:10.160 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-16T23:57:10.160 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:57:10.160 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T23:57:10.232 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T23:57:10.342 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:57:10.342 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T23:57:10.374 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T23:57:10.445 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:57:10.445 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T23:57:10.598 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T23:57:10.628 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-16T23:57:10.629 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:57:10.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T23:57:10.699 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T23:57:10.808 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:57:10.809 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T23:57:10.840 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T23:57:10.910 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:57:10.910 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T23:57:10.943 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T23:57:11.014 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-16T23:57:11.014 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:57:11.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T23:57:11.083 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T23:57:11.193 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:57:11.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T23:57:11.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T23:57:11.296 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:57:11.297 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T23:57:11.329 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T23:57:11.400 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-16T23:57:11.474 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'subvols': {'create': 2}} 2023-11-16T23:57:11.475 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:11.475 INFO:tasks.cephadm:Cluster fsid is dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:57:11.475 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-16T23:57:11.476 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.6', 'mon.b': '172.21.15.87', 'mon.c': '172.21.15.186'} 2023-11-16T23:57:11.476 INFO:tasks.cephadm:First mon is mon.a on smithi006 2023-11-16T23:57:11.476 INFO:tasks.cephadm:First mgr is x 2023-11-16T23:57:11.476 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-16T23:57:11.476 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2023-11-16T23:57:11.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname $(hostname -s) 2023-11-16T23:57:11.582 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2023-11-16T23:57:11.614 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-16T23:57:11.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:11.827 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74215/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3906-g1c956514', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3906.g1c956514'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 20:52:31.578811', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-16T23:57:11.962 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-choffman-62793, sha1 1c9565146c8647d4e00eb6c7ae626dc192680e7b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-16T23:57:11.962 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm 2023-11-16T23:57:11.962 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 23:57 /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:12.359 DEBUG:teuthology.orchestra.run.smithi087:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:12.890 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 23:57 /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:12.952 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:13.352 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 23:57 /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:13.353 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:13.404 DEBUG:teuthology.orchestra.run.smithi087:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:13.426 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T23:57:13.465 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b on all hosts... 2023-11-16T23:57:13.466 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-16T23:57:13.470 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-16T23:57:13.477 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-16T23:57:14.828 INFO:teuthology.orchestra.run.smithi087.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-16T23:57:14.898 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-16T23:57:17.928 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-16T23:57:40.575 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-11-16T23:57:40.575 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-16T23:57:40.575 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-16T23:57:40.575 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2023-11-16T23:57:40.575 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-16T23:57:40.576 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2023-11-16T23:57:40.576 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-11-16T23:57:45.109 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-11-16T23:57:45.109 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-16T23:57:45.109 INFO:teuthology.orchestra.run.smithi087.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-16T23:57:45.109 INFO:teuthology.orchestra.run.smithi087.stdout: "repo_digests": [ 2023-11-16T23:57:45.109 INFO:teuthology.orchestra.run.smithi087.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-16T23:57:45.110 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2023-11-16T23:57:45.110 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-11-16T23:57:49.107 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-16T23:57:49.108 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-16T23:57:49.134 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2023-11-16T23:57:49.210 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph 2023-11-16T23:57:49.245 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2023-11-16T23:57:49.283 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2023-11-16T23:57:49.353 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /etc/ceph 2023-11-16T23:57:49.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2023-11-16T23:57:49.415 INFO:tasks.cephadm:Writing seed config... 2023-11-16T23:57:49.416 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-16T23:57:49.416 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-16T23:57:49.417 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-16T23:57:49.418 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-16T23:57:49.419 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-16T23:57:49.420 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-16T23:57:49.420 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-16T23:57:49.420 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-16T23:57:49.420 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-11-16T23:57:49.420 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:57:49.421 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-16T23:57:49.466 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-16T23:57:49.467 DEBUG:teuthology.orchestra.run.smithi006:mon.a> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service 2023-11-16T23:57:49.511 DEBUG:teuthology.orchestra.run.smithi006:mgr.x> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x.service 2023-11-16T23:57:49.555 INFO:tasks.cephadm:Bootstrapping... 2023-11-16T23:57:49.555 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b -v bootstrap --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-16T23:57:49.579 INFO:journalctl@ceph.mon.a.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-16T23:57:49.624 INFO:journalctl@ceph.mgr.x.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-16T23:57:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-16T23:57:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b', '-v', 'bootstrap', '--fsid', 'dbc2f8cc-84db-11ee-8dbe-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2023-11-16T23:57:50.086 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.1.1 2023-11-16T23:57:50.087 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-16T23:57:50.087 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2023-11-16T23:57:50.259 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.1.1 2023-11-16T23:57:50.259 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2023-11-16T23:57:50.259 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2023-11-16T23:57:50.274 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T23:57:50.274 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T23:57:50.287 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T23:57:50.288 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:57:50.301 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2023-11-16T23:57:50.314 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2023-11-16T23:57:50.315 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2023-11-16T23:57:50.315 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2023-11-16T23:57:50.487 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.1.1 2023-11-16T23:57:50.487 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-16T23:57:50.488 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2023-11-16T23:57:50.488 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2023-11-16T23:57:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T23:57:50.501 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T23:57:50.514 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T23:57:50.515 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:57:50.528 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2023-11-16T23:57:50.542 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2023-11-16T23:57:50.542 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2023-11-16T23:57:50.542 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2023-11-16T23:57:50.542 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:57:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 139886376921576 on /run/cephadm/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.lock 2023-11-16T23:57:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 139886376921576 acquired on /run/cephadm/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.lock 2023-11-16T23:57:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2023-11-16T23:57:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2023-11-16T23:57:50.543 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2023-11-16T23:57:50.551 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-11-16T23:57:50.551 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2023-11-16T23:57:50.558 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-16T23:57:50.558 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-11-16T23:57:50.558 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1776sec hoplimit 64 pref medium 2023-11-16T23:57:50.565 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-16T23:57:50.565 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-16T23:57:50.565 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T23:57:50.565 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-11-16T23:57:50.565 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-16T23:57:50.566 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T23:57:50.566 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2023-11-16T23:57:50.566 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2023-11-16T23:57:50.566 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-16T23:57:50.567 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-16T23:57:50.568 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:eb1dc24ed1b729f25bfe21673f4e5cc46a126932dc459841b1c31fbebaca605e 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-16T23:57:51.044 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-16T23:57:51.045 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-16T23:57:51.045 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Storing signatures 2023-11-16T23:57:52.123 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-16T23:57:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-16T23:57:52.124 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2023-11-16T23:57:53.104 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2023-11-16T23:57:53.104 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2023-11-16T23:57:53.982 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCBrFZlSg32HxAAHQu13NbGsBFeaLerGn8imA== 2023-11-16T23:57:54.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCCrFZlR4kDGhAAxKQ1xfafCz39NjHWKFdzlQ== 2023-11-16T23:57:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCDrFZl/734FxAAPlPrtU+olHwwgpb5oPO6jg== 2023-11-16T23:57:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2023-11-16T23:57:57.020 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for a [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = pacific 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:57:57.163 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2023-11-16T23:57:58.104 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.a on 2023-11-16T23:57:58.325 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-16T23:57:58.535 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-16T23:57:58.762 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target. 2023-11-16T23:57:58.763 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target. 2023-11-16T23:57:59.064 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a 2023-11-16T23:57:59.065 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service: Unit ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service not loaded. 2023-11-16T23:57:59.271 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target.wants/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T23:57:59.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:57:59 smithi006 systemd[1]: Starting Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:58:00.181 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:57:59 smithi006 podman[97294]: 2023-11-16T23:58:00.298 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T23:58:00.298 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2023-11-16T23:58:00.315 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2023-11-16T23:58:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:00 smithi006 bash[97294]: b1881d4d2c78da3fc8419f3265b899e2b3ff7a1275601faf92f723960edca71c 2023-11-16T23:58:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:00 smithi006 ceph-mon[97335]: mkfs dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:00 smithi006 ceph-mon[97335]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T23:58:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:00 smithi006 systemd[1]: Started Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: monmap e1: 1 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: fsmap 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: mgrmap e1: no daemons active 2023-11-16T23:58:01.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:01 smithi006 ceph-mon[97335]: from='client.? 172.21.15.6:0/2845741597' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-16T23:58:03.108 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.783026s) 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-16T23:58:03.109 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.913+0000 7efda73f0700 1 Processor -- start 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.913+0000 7efda73f0700 1 -- start start 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.914+0000 7efda73f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.914+0000 7efda73f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01062a0 con 0x7efda0105950 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.915+0000 7efda518c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.915+0000 7efda518c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35810/0 (socket says 172.21.15.6:35810) 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.915+0000 7efda518c700 1 -- 172.21.15.6:0/4116342351 learned_addr learned my addr 172.21.15.6:0/4116342351 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.915+0000 7efda518c700 1 -- 172.21.15.6:0/4116342351 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda01063e0 con 0x7efda0105950 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.915+0000 7efda518c700 1 --2- 172.21.15.6:0/4116342351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098a0 tx=0x7efd9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4523ba4e44ab1210 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:03.110 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efd97fff700 1 -- 172.21.15.6:0/4116342351 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c004160 con 0x7efda0105950 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efd97fff700 1 -- 172.21.15.6:0/4116342351 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c0042c0 con 0x7efda0105950 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efd97fff700 1 -- 172.21.15.6:0/4116342351 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c0045a0 con 0x7efda0105950 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 -- 172.21.15.6:0/4116342351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 msgr2=0x7efda0105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 --2- 172.21.15.6:0/4116342351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098a0 tx=0x7efd9c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 -- 172.21.15.6:0/4116342351 shutdown_connections 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 --2- 172.21.15.6:0/4116342351 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0105d60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 -- 172.21.15.6:0/4116342351 >> 172.21.15.6:0/4116342351 conn(0x7efda0100fe0 msgr2=0x7efda01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 -- 172.21.15.6:0/4116342351 shutdown_connections 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.916+0000 7efda73f0700 1 -- 172.21.15.6:0/4116342351 wait complete. 2023-11-16T23:58:03.111 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.917+0000 7efda73f0700 1 Processor -- start 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.917+0000 7efda73f0700 1 -- start start 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.918+0000 7efda73f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.918+0000 7efda73f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0199640 con 0x7efda0105950 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.918+0000 7efda518c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.918+0000 7efda518c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35824/0 (socket says 172.21.15.6:35824) 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.918+0000 7efda518c700 1 -- 172.21.15.6:0/2845741597 learned_addr learned my addr 172.21.15.6:0/2845741597 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efda518c700 1 -- 172.21.15.6:0/2845741597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009550 con 0x7efda0105950 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efda518c700 1 --2- 172.21.15.6:0/2845741597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efda0106680 tx=0x7efd9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c011630 con 0x7efda0105950 2023-11-16T23:58:03.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c011790 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda0106ea0 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c011a30 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda0106d10 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.919+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efd9c022070 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.920+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd9c01c790 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.920+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda0192d00 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:00.923+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7efd9c01bcf0 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.000+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7efda0062120 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.000+0000 7efd967fc700 1 -- 172.21.15.6:0/2845741597 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7efda0062120 con 0x7efda0105950 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 msgr2=0x7efda0199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 --2- 172.21.15.6:0/2845741597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efda0106680 tx=0x7efd9c004750 comp rx=0 tx=0).stop 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 shutdown_connections 2023-11-16T23:58:03.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 --2- 172.21.15.6:0/2845741597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efda0105950 0x7efda0199100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:03.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 >> 172.21.15.6:0/2845741597 conn(0x7efda0100fe0 msgr2=0x7efda018fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:03.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 shutdown_connections 2023-11-16T23:58:03.114 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:01.001+0000 7efda73f0700 1 -- 172.21.15.6:0/2845741597 wait complete. 2023-11-16T23:58:03.114 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2023-11-16T23:58:03.114 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2023-11-16T23:58:04.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:03 smithi006 ceph-mon[97335]: from='client.? 172.21.15.6:0/3817567734' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T23:58:04.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:03 smithi006 ceph-mon[97335]: from='client.? 172.21.15.6:0/3817567734' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-16T23:58:04.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:04.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T23:58:04.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:04.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2023-11-16T23:58:04.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T23:58:04.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.695+0000 7f00c838f700 1 Processor -- start 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.695+0000 7f00c838f700 1 -- start start 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.696+0000 7f00c838f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.696+0000 7f00c838f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01062a0 con 0x7f00c0105950 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.696+0000 7f00c612b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:04.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.696+0000 7f00c612b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35826/0 (socket says 172.21.15.6:35826) 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.696+0000 7f00c612b700 1 -- 172.21.15.6:0/1334324166 learned_addr learned my addr 172.21.15.6:0/1334324166 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.697+0000 7f00c612b700 1 -- 172.21.15.6:0/1334324166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c01063e0 con 0x7f00c0105950 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.697+0000 7f00c612b700 1 --2- 172.21.15.6:0/1334324166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098a0 tx=0x7f00bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c719a2a1aba503e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.697+0000 7f00c5129700 1 -- 172.21.15.6:0/1334324166 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00bc004160 con 0x7f00c0105950 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.697+0000 7f00c5129700 1 -- 172.21.15.6:0/1334324166 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0042c0 con 0x7f00c0105950 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.697+0000 7f00c5129700 1 -- 172.21.15.6:0/1334324166 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00bc0045a0 con 0x7f00c0105950 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 -- 172.21.15.6:0/1334324166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 msgr2=0x7f00c0105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 --2- 172.21.15.6:0/1334324166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098a0 tx=0x7f00bc009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:04.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 -- 172.21.15.6:0/1334324166 shutdown_connections 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 --2- 172.21.15.6:0/1334324166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c0105d60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 -- 172.21.15.6:0/1334324166 >> 172.21.15.6:0/1334324166 conn(0x7f00c0100fe0 msgr2=0x7f00c01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 -- 172.21.15.6:0/1334324166 shutdown_connections 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.698+0000 7f00c838f700 1 -- 172.21.15.6:0/1334324166 wait complete. 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.699+0000 7f00c838f700 1 Processor -- start 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.699+0000 7f00c838f700 1 -- start start 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.699+0000 7f00c838f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.699+0000 7f00c838f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c019b840 con 0x7f00c0105950 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00c612b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00c612b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35838/0 (socket says 172.21.15.6:35838) 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00c612b700 1 -- 172.21.15.6:0/3817567734 learned_addr learned my addr 172.21.15.6:0/3817567734 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:04.403 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00c612b700 1 -- 172.21.15.6:0/3817567734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009550 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00c612b700 1 --2- 172.21.15.6:0/3817567734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f00c0106680 tx=0x7f00bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00bc011630 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc011790 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.700+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00bc011a30 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.701+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c0106ea0 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.701+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c0106d10 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.701+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f00bc022070 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.701+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00bc01c450 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.702+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c0107030 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.704+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f00bc01b450 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.777+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f00c0062120 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.781+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00bc01b5f0 con 0x7f00c0105950 2023-11-16T23:58:04.404 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.781+0000 7f00b77fe700 1 -- 172.21.15.6:0/3817567734 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f00c0062120 con 0x7f00c0105950 2023-11-16T23:58:04.405 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.782+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 msgr2=0x7f00c019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:04.405 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.782+0000 7f00c838f700 1 --2- 172.21.15.6:0/3817567734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f00c0106680 tx=0x7f00bc004750 comp rx=0 tx=0).stop 2023-11-16T23:58:04.405 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.782+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 shutdown_connections 2023-11-16T23:58:04.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.782+0000 7f00c838f700 1 --2- 172.21.15.6:0/3817567734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00c0105950 0x7f00c019b300 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:04.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.782+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 >> 172.21.15.6:0/3817567734 conn(0x7f00c0100fe0 msgr2=0x7f00c007a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:04.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.783+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 shutdown_connections 2023-11-16T23:58:04.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:03.783+0000 7f00c838f700 1 -- 172.21.15.6:0/3817567734 wait complete. 2023-11-16T23:58:04.408 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2023-11-16T23:58:05.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:05 smithi006 ceph-mon[97335]: from='client.? 172.21.15.6:0/2108329006' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:58:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.959+0000 7f72a521a700 1 Processor -- start 2023-11-16T23:58:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.959+0000 7f72a521a700 1 -- start start 2023-11-16T23:58:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.959+0000 7f72a521a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.959+0000 7f72a521a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a007a820 con 0x7f72a007bde0 2023-11-16T23:58:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.960+0000 7f729ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:05.586 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.960+0000 7f729ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35852/0 (socket says 172.21.15.6:35852) 2023-11-16T23:58:05.586 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.960+0000 7f729ed9d700 1 -- 172.21.15.6:0/2488692227 learned_addr learned my addr 172.21.15.6:0/2488692227 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:05.586 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.960+0000 7f729ed9d700 1 -- 172.21.15.6:0/2488692227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a007a960 con 0x7f72a007bde0 2023-11-16T23:58:05.586 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.960+0000 7f729ed9d700 1 --2- 172.21.15.6:0/2488692227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999377fadb094da server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:05.586 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f729dd9b700 1 -- 172.21.15.6:0/2488692227 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7290004160 con 0x7f72a007bde0 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f729dd9b700 1 -- 172.21.15.6:0/2488692227 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72900042c0 con 0x7f72a007bde0 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f729dd9b700 1 -- 172.21.15.6:0/2488692227 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7290011a00 con 0x7f72a007bde0 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f72a521a700 1 -- 172.21.15.6:0/2488692227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 msgr2=0x7f72a007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f72a521a700 1 --2- 172.21.15.6:0/2488692227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f72a521a700 1 -- 172.21.15.6:0/2488692227 shutdown_connections 2023-11-16T23:58:05.587 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f72a521a700 1 --2- 172.21.15.6:0/2488692227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a007a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.961+0000 7f72a521a700 1 -- 172.21.15.6:0/2488692227 >> 172.21.15.6:0/2488692227 conn(0x7f72a0100fe0 msgr2=0x7f72a01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.962+0000 7f72a521a700 1 -- 172.21.15.6:0/2488692227 shutdown_connections 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.962+0000 7f72a521a700 1 -- 172.21.15.6:0/2488692227 wait complete. 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f72a521a700 1 Processor -- start 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f72a521a700 1 -- start start 2023-11-16T23:58:05.588 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f72a521a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f72a521a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019c050 con 0x7f72a007bde0 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f729ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f729ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35868/0 (socket says 172.21.15.6:35868) 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.963+0000 7f729ed9d700 1 -- 172.21.15.6:0/2108329006 learned_addr learned my addr 172.21.15.6:0/2108329006 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f729ed9d700 1 -- 172.21.15.6:0/2108329006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290009550 con 0x7f72a007bde0 2023-11-16T23:58:05.589 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f729ed9d700 1 --2- 172.21.15.6:0/2108329006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7290006af0 tx=0x7f7290011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7290003a40 con 0x7f72a007bde0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7290003ba0 con 0x7f72a007bde0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f729001a700 con 0x7f72a007bde0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a010be00 con 0x7f72a007bde0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.964+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a010bc70 con 0x7f72a007bde0 2023-11-16T23:58:05.590 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.965+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f729001a930 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.965+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f729002dd10 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.966+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0195960 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:04.969+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f72900248d0 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.042+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f72a010bf90 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.043+0000 7f7287fff700 1 -- 172.21.15.6:0/2108329006 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f729001ad20 con 0x7f72a007bde0 2023-11-16T23:58:05.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 msgr2=0x7f72a019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 --2- 172.21.15.6:0/2108329006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7290006af0 tx=0x7f7290011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 shutdown_connections 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 --2- 172.21.15.6:0/2108329006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72a007bde0 0x7f72a019bb10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 >> 172.21.15.6:0/2108329006 conn(0x7f72a0100fe0 msgr2=0x7f72a0101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 shutdown_connections 2023-11-16T23:58:05.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:05.044+0000 7f72a521a700 1 -- 172.21.15.6:0/2108329006 wait complete. 2023-11-16T23:58:05.593 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2023-11-16T23:58:05.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:05 smithi006 systemd[1]: Stopping Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:58:06.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:06 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a[97310]: 2023-11-16T23:58:06.084+0000 7f30cf1ea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T23:58:06.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:06 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a[97310]: 2023-11-16T23:58:06.084+0000 7f30cf1ea700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-16T23:58:06.885 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:06 smithi006 bash[97672]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a 2023-11-16T23:58:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service: Succeeded. 2023-11-16T23:58:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 systemd[1]: Stopped Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:58:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 systemd[1]: Starting Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:58:07.797 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 podman[97800]: 2023-11-16T23:58:07.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 bash[97800]: c70c8f5ff36cb4cb64b202ee99b303a3749a8936f21932beeec6c29ff84933b4 2023-11-16T23:58:07.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi006.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-16T23:58:08.131 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: pidfile_write: ignore empty --pid-file 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: load: jerasure load: lrc 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: RocksDB version: 7.9.2 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Git sha 0 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: DB SUMMARY 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: DB Session ID: M2RYKP64RAMGWJU6DTMD 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: CURRENT file: CURRENT 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98660 ; 2023-11-16T23:58:08.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.error_if_exists: 0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.create_if_missing: 0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.paranoid_checks: 1 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.env: 0x563effd66720 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.info_log: 0x563f02c1d3c0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.statistics: (nil) 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.use_fsync: 0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_log_file_size: 0 2023-11-16T23:58:08.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_fallocate: 1 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.use_direct_reads: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.db_log_dir: 2023-11-16T23:58:08.134 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.wal_dir: 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.write_buffer_manager: 0x563f01e8e3c0 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T23:58:08.135 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.unordered_write: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.row_cache: None 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.wal_filter: None 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T23:58:08.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.two_write_queues: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.wal_compression: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.atomic_flush: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.log_readahead_size: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T23:58:08.137 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_background_jobs: 2 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_background_compactions: -1 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_subcompactions: 1 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T23:58:08.139 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_open_files: -1 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_background_flushes: -1 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Compression algorithms supported: 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kZSTD supported: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kXpressCompression supported: 0 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kZlibCompression supported: 1 2023-11-16T23:58:08.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kSnappyCompression supported: 1 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kLZ4Compression supported: 1 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: kBZip2Compression supported: 0 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.merge_operator: 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_filter: None 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T23:58:08.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563f02c1d4e0) 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks: 1 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_top_level_index_and_filter: 1 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_type: 0 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_index_type: 0 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_shortening: 1 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: checksum: 4 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: no_block_cache: 0 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache: 0x563f01f2f350 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_name: BinnedLRUCache 2023-11-16T23:58:08.142 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_options: 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: capacity : 536870912 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_shard_bits : 4 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: strict_capacity_limit : 0 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: high_pri_pool_ratio: 0.000 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_compressed: (nil) 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: persistent_cache: (nil) 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size: 4096 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size_deviation: 10 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_restart_interval: 16 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_block_restart_interval: 1 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: metadata_block_size: 4096 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: partition_filters: 0 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: use_delta_encoding: 1 2023-11-16T23:58:08.143 INFO:journalctl@ceph.mon.a.smithi006.stdout: filter_policy: bloomfilter 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: whole_key_filtering: 1 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: verify_compression: 0 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: read_amp_bytes_per_bit: 0 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: format_version: 5 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_index_compression: 1 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_align: 0 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: max_auto_readahead_size: 262144 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: prepopulate_block_cache: 0 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: initial_auto_readahead_size: 8192 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression: NoCompression 2023-11-16T23:58:08.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.num_levels: 7 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T23:58:08.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T23:58:08.146 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T23:58:08.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T23:58:08.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.table_properties_collectors: 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 systemd[1]: Started Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.inplace_update_support: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.bloom_locality: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.max_successive_merges: 0 2023-11-16T23:58:08.149 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.ttl: 2592000 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enable_blob_files: false 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.min_blob_size: 0 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T23:58:08.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f862be94-a24f-45a9-b4a9-7c2d6ff8afb7 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179087818493, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179087820304, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93689, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 321, "table_properties": {"data_size": 91699, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13541, "raw_average_key_size": 49, "raw_value_size": 84319, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 272, "num_filter_entries": 272, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700179087, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f862be94-a24f-45a9-b4a9-7c2d6ff8afb7", "db_session_id": "M2RYKP64RAMGWJU6DTMD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179087820457, "job": 1, "event": "recovery_finished"} 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T23:58:08.151 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563f01fcc000 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: DB pointer 0x563f01fb8000 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** DB Stats ** 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2023-11-16T23:58:08.152 INFO:journalctl@ceph.mon.a.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: L0 2/0 93.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: Sum 2/0 93.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 81.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.153 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative compaction: 0.00 GB write, 10.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval compaction: 0.00 GB write, 10.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache BinnedLRUCache@0x563f01f2f350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: starting mon.a rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:08.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???) e1 preinit fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).mds e1 new map 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).mds e1 print_map 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: e1 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: legacy client fscid: -1 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout: No filesystems configured 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:58:08.155 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expand map: {default=false} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta from 'false' to 'false' 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expanded map: {default=false} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expand map: {default=info} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta from 'info' to 'info' 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expanded map: {default=info} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expand map: {default=daemon} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expand map: {default=debug} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: expand_channel_meta expanded map: {default=debug} 2023-11-16T23:58:08.156 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T23:58:08.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: monmap e1: 1 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} 2023-11-16T23:58:08.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: fsmap 2023-11-16T23:58:08.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T23:58:08.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:07 smithi006 ceph-mon[97840]: mgrmap e1: no daemons active 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.422+0000 7f79e51d3700 1 Processor -- start 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.422+0000 7f79e51d3700 1 -- start start 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.423+0000 7f79e51d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.423+0000 7f79e51d3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e007a7e0 con 0x7f79e007bda0 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.423+0000 7f79ded9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.423+0000 7f79ded9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35884/0 (socket says 172.21.15.6:35884) 2023-11-16T23:58:09.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.423+0000 7f79ded9d700 1 -- 172.21.15.6:0/2804951650 learned_addr learned my addr 172.21.15.6:0/2804951650 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.424+0000 7f79ded9d700 1 -- 172.21.15.6:0/2804951650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e007a920 con 0x7f79e007bda0 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.424+0000 7f79ded9d700 1 --2- 172.21.15.6:0/2804951650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79d00098a0 tx=0x7f79d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60293f0d67318ba3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.424+0000 7f79ddd9b700 1 -- 172.21.15.6:0/2804951650 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79d0004160 con 0x7f79e007bda0 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.424+0000 7f79ddd9b700 1 -- 172.21.15.6:0/2804951650 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79d00042c0 con 0x7f79e007bda0 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.424+0000 7f79ddd9b700 1 -- 172.21.15.6:0/2804951650 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79d0011a00 con 0x7f79e007bda0 2023-11-16T23:58:09.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 -- 172.21.15.6:0/2804951650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 msgr2=0x7f79e007a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 --2- 172.21.15.6:0/2804951650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79d00098a0 tx=0x7f79d0009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 -- 172.21.15.6:0/2804951650 shutdown_connections 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 --2- 172.21.15.6:0/2804951650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e007a2a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 -- 172.21.15.6:0/2804951650 >> 172.21.15.6:0/2804951650 conn(0x7f79e0100fe0 msgr2=0x7f79e01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 -- 172.21.15.6:0/2804951650 shutdown_connections 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.425+0000 7f79e51d3700 1 -- 172.21.15.6:0/2804951650 wait complete. 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.426+0000 7f79e51d3700 1 Processor -- start 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79e51d3700 1 -- start start 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79e51d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79e51d3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e019c030 con 0x7f79e007bda0 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79ded9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:09.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79ded9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35896/0 (socket says 172.21.15.6:35896) 2023-11-16T23:58:09.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.427+0000 7f79ded9d700 1 -- 172.21.15.6:0/400429395 learned_addr learned my addr 172.21.15.6:0/400429395 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:09.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79ded9d700 1 -- 172.21.15.6:0/400429395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009550 con 0x7f79e007bda0 2023-11-16T23:58:09.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79ded9d700 1 --2- 172.21.15.6:0/400429395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79d0006af0 tx=0x7f79d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:09.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79d0003a40 con 0x7f79e007bda0 2023-11-16T23:58:09.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79d0003ba0 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e010be40 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79d001a700 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.428+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e010bcb0 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.429+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f79d001ac80 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.429+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f79d0019700 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.430+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e010bfd0 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.433+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f79d0024ad0 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.505+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f79e004e8c0 con 0x7f79e007bda0 2023-11-16T23:58:09.015 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.508+0000 7f79c7fff700 1 -- 172.21.15.6:0/400429395 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f79d0024c70 con 0x7f79e007bda0 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.509+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 msgr2=0x7f79e019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.509+0000 7f79e51d3700 1 --2- 172.21.15.6:0/400429395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79d0006af0 tx=0x7f79d0011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.509+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 shutdown_connections 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.509+0000 7f79e51d3700 1 --2- 172.21.15.6:0/400429395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79e007bda0 0x7f79e019baf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.509+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 >> 172.21.15.6:0/400429395 conn(0x7f79e0100fe0 msgr2=0x7f79e0101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.510+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 shutdown_connections 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:08.510+0000 7f79e51d3700 1 -- 172.21.15.6:0/400429395 wait complete. 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2023-11-16T23:58:09.016 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 9283 ... 2023-11-16T23:58:09.017 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 8765 ... 2023-11-16T23:58:09.326 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-16T23:58:09.326 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x.service: Unit ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x.service not loaded. 2023-11-16T23:58:09.551 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.target.wants/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T23:58:09.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/400429395' entity='client.admin' 2023-11-16T23:58:09.724 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:09 smithi006 systemd[1]: Starting Ceph mgr.x for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:58:10.299 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 podman[98070]: 2023-11-16T23:58:10.471 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T23:58:10.471 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2023-11-16T23:58:10.488 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T23:58:10.488 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:58:10.488 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2023-11-16T23:58:10.488 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T23:58:10.506 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T23:58:10.507 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2023-11-16T23:58:10.525 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2023-11-16T23:58:10.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 bash[98070]: 31f93bc4dda322246166a92bbceb947f3bd045fb97f92520f8e43d7e635292be 2023-11-16T23:58:10.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 systemd[1]: Started Ceph mgr.x for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:58:10.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:10.620+0000 7f7970e38200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T23:58:10.997 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:10.729+0000 7f7970e38200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T23:58:10.997 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:10.861+0000 7f7970e38200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T23:58:10.997 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:10.995+0000 7f7970e38200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T23:58:11.256 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:11.230+0000 7f7970e38200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T23:58:11.671 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:11 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/435658429' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T23:58:11.722 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:11.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:58:11.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "dbc2f8cc-84db-11ee-8dbe-212e2dc638e7", 2023-11-16T23:58:11.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T23:58:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:11.752 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T23:58:11.758 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T23:58:11.759 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T23:58:11.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T23:58:11.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T23:58:11.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T23:58:11.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T23:58:11.760 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T23:58:11.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T23:58:11.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:11.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T23:58:11.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T23:58:11.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:11.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T23:58:00.220752+0000", 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.144+0000 7fba4f209700 1 Processor -- start 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.145+0000 7fba4f209700 1 -- start start 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.145+0000 7fba4f209700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.145+0000 7fba4f209700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4807a7e0 con 0x7fba4807bda0 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba4cfa5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba4cfa5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47896/0 (socket says 172.21.15.6:47896) 2023-11-16T23:58:11.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba4cfa5700 1 -- 172.21.15.6:0/822725670 learned_addr learned my addr 172.21.15.6:0/822725670 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba4cfa5700 1 -- 172.21.15.6:0/822725670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4807a920 con 0x7fba4807bda0 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba4cfa5700 1 --2- 172.21.15.6:0/822725670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0098a0 tx=0x7fba3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a28e0739c6adf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.146+0000 7fba477fe700 1 -- 172.21.15.6:0/822725670 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba3c004160 con 0x7fba4807bda0 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba477fe700 1 -- 172.21.15.6:0/822725670 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba3c0042c0 con 0x7fba4807bda0 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba477fe700 1 -- 172.21.15.6:0/822725670 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba3c011a00 con 0x7fba4807bda0 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 -- 172.21.15.6:0/822725670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 msgr2=0x7fba4807a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 --2- 172.21.15.6:0/822725670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0098a0 tx=0x7fba3c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 -- 172.21.15.6:0/822725670 shutdown_connections 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 --2- 172.21.15.6:0/822725670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4807a2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 -- 172.21.15.6:0/822725670 >> 172.21.15.6:0/822725670 conn(0x7fba48100fe0 msgr2=0x7fba481033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:11.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.147+0000 7fba4f209700 1 -- 172.21.15.6:0/822725670 shutdown_connections 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.148+0000 7fba4f209700 1 -- 172.21.15.6:0/822725670 wait complete. 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4f209700 1 Processor -- start 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4f209700 1 -- start start 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4f209700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4f209700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4819bfc0 con 0x7fba4807bda0 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4cfa5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4cfa5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47908/0 (socket says 172.21.15.6:47908) 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.149+0000 7fba4cfa5700 1 -- 172.21.15.6:0/435658429 learned_addr learned my addr 172.21.15.6:0/435658429 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba4cfa5700 1 -- 172.21.15.6:0/435658429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c009550 con 0x7fba4807bda0 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba4cfa5700 1 --2- 172.21.15.6:0/435658429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009fb0 tx=0x7fba3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba3c003930 con 0x7fba4807bda0 2023-11-16T23:58:11.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba3c003a90 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba3c0195e0 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.150+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4810be40 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.151+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4810bcb0 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.151+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba3c019b50 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.151+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba3c01a430 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.152+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4810bfd0 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.154+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fba3c003c00 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.231+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fba48062120 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.232+0000 7fba45ffb700 1 -- 172.21.15.6:0/435658429 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fba3c02de30 con 0x7fba4807bda0 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 msgr2=0x7fba4819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 --2- 172.21.15.6:0/435658429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009fb0 tx=0x7fba3c011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 shutdown_connections 2023-11-16T23:58:11.787 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 --2- 172.21.15.6:0/435658429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba4807bda0 0x7fba4819ba80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:11.788 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 >> 172.21.15.6:0/435658429 conn(0x7fba48100fe0 msgr2=0x7fba48101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:11.788 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 shutdown_connections 2023-11-16T23:58:11.788 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:11.233+0000 7fba4f209700 1 -- 172.21.15.6:0/435658429 wait complete. 2023-11-16T23:58:11.788 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2023-11-16T23:58:11.968 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:11.670+0000 7f7970e38200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T23:58:11.968 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:11.798+0000 7f7970e38200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T23:58:12.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:11.966+0000 7f7970e38200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T23:58:12.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:12.094+0000 7f7970e38200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T23:58:12.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:12.660+0000 7f7970e38200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T23:58:13.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:13.472+0000 7f7970e38200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T23:58:13.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:13.687+0000 7f7970e38200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T23:58:14.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:14.694+0000 7f7970e38200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:58:14.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:14.818+0000 7f7970e38200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T23:58:15.423 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:15.047+0000 7f7970e38200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:58:15.708 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:15.421+0000 7f7970e38200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T23:58:15.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:15.707+0000 7f7970e38200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T23:58:15.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:15.905+0000 7f7970e38200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T23:58:16.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:16.259+0000 7f7970e38200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T23:58:17.662 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:17.226+0000 7f7970e38200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T23:58:17.662 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:17.404+0000 7f7970e38200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T23:58:17.662 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:17.660+0000 7f7970e38200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T23:58:17.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:17.814+0000 7f7970e38200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T23:58:18.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:18.100+0000 7f7970e38200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T23:58:19.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:19.213+0000 7f7970e38200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T23:58:19.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: Activating manager daemon x 2023-11-16T23:58:19.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: mgrmap e2: x(active, starting, since 0.00406086s) 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: Manager daemon x is now available 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' 2023-11-16T23:58:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:19 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' 2023-11-16T23:58:20.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:20 smithi006 ceph-mon[97840]: from='mgr.14100 172.21.15.6:0/386428293' entity='mgr.x' 2023-11-16T23:58:20.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:20 smithi006 ceph-mon[97840]: mgrmap e3: x(active, since 1.00986s) 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "dbc2f8cc-84db-11ee-8dbe-212e2dc638e7", 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T23:58:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T23:58:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T23:58:21.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.948 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T23:58:21.950 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:21.950 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T23:58:21.950 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T23:58:21.950 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T23:58:21.961 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T23:58:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T23:58:00.220752+0000", 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.056+0000 7f611556d700 1 Processor -- start 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.056+0000 7f611556d700 1 -- start start 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.056+0000 7f611556d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.056+0000 7f611556d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611007a7c0 con 0x7f611007bd80 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.057+0000 7f610effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.057+0000 7f610effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33710/0 (socket says 172.21.15.6:33710) 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.057+0000 7f610effd700 1 -- 172.21.15.6:0/2483295952 learned_addr learned my addr 172.21.15.6:0/2483295952 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.057+0000 7f610effd700 1 -- 172.21.15.6:0/2483295952 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611007a900 con 0x7f611007bd80 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.057+0000 7f610effd700 1 --2- 172.21.15.6:0/2483295952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61000098a0 tx=0x7f6100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0cee0be58db250 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f610dffb700 1 -- 172.21.15.6:0/2483295952 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6100004160 con 0x7f611007bd80 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f610dffb700 1 -- 172.21.15.6:0/2483295952 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61000042c0 con 0x7f611007bd80 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f610dffb700 1 -- 172.21.15.6:0/2483295952 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6100011a00 con 0x7f611007bd80 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 -- 172.21.15.6:0/2483295952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 msgr2=0x7f611007a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 --2- 172.21.15.6:0/2483295952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61000098a0 tx=0x7f6100009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 -- 172.21.15.6:0/2483295952 shutdown_connections 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 --2- 172.21.15.6:0/2483295952 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611007a280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 -- 172.21.15.6:0/2483295952 >> 172.21.15.6:0/2483295952 conn(0x7f6110100fe0 msgr2=0x7f61101033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 -- 172.21.15.6:0/2483295952 shutdown_connections 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.058+0000 7f611556d700 1 -- 172.21.15.6:0/2483295952 wait complete. 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.060+0000 7f611556d700 1 Processor -- start 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.060+0000 7f611556d700 1 -- start start 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f611556d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f611556d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611019c030 con 0x7f611007bd80 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f610effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f610effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33726/0 (socket says 172.21.15.6:33726) 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f610effd700 1 -- 172.21.15.6:0/4019529314 learned_addr learned my addr 172.21.15.6:0/4019529314 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f610effd700 1 -- 172.21.15.6:0/4019529314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6100009550 con 0x7f611007bd80 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.061+0000 7f610effd700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6100006af0 tx=0x7f6100011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.062+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6100003a40 con 0x7f611007bd80 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.062+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6100003ba0 con 0x7f611007bd80 2023-11-16T23:58:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.062+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610001a700 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.062+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611010be00 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.062+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611010bc70 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.063+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6110062120 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.063+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f6100024ad0 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.063+0000 7f60f7fff700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 0x7f60f803ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.063+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f610004c470 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.064+0000 7f610e7fc700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 0x7f60f803ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.064+0000 7f610e7fc700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 0x7f60f803ab50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6104006fd0 tx=0x7f6104006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.069+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6100018bd0 con 0x7f611007bd80 2023-11-16T23:58:21.966 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.279+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f610002d770 con 0x7f611007bd80 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.357+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f611004e8c0 con 0x7f611007bd80 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.358+0000 7f60f7fff700 1 -- 172.21.15.6:0/4019529314 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6100018e00 con 0x7f611007bd80 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 msgr2=0x7f60f803ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 0x7f60f803ab50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6104006fd0 tx=0x7f6104006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 msgr2=0x7f611019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6100006af0 tx=0x7f6100011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 shutdown_connections 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f60f80386a0 0x7f60f803ab50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 --2- 172.21.15.6:0/4019529314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f611007bd80 0x7f611019baf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 >> 172.21.15.6:0/4019529314 conn(0x7f6110100fe0 msgr2=0x7f6110101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 shutdown_connections 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:21.360+0000 7f611556d700 1 -- 172.21.15.6:0/4019529314 wait complete. 2023-11-16T23:58:21.967 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2023-11-16T23:58:22.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:22 smithi006 ceph-mon[97840]: mgrmap e4: x(active, since 2s) 2023-11-16T23:58:22.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4019529314' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T23:58:23.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c62a46700 1 Processor -- start 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c62a46700 1 -- start start 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c62a46700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:23.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c62a46700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c07a7c0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33738/0 (socket says 172.21.15.6:33738) 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.566+0000 7f0c5bfff700 1 -- 172.21.15.6:0/1830249930 learned_addr learned my addr 172.21.15.6:0/1830249930 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.567+0000 7f0c5bfff700 1 -- 172.21.15.6:0/1830249930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c07a900 con 0x7f0c5c07bd80 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.567+0000 7f0c5bfff700 1 --2- 172.21.15.6:0/1830249930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0098a0 tx=0x7f0c4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba1d6454d70f25e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.567+0000 7f0c5affd700 1 -- 172.21.15.6:0/1830249930 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c004160 con 0x7f0c5c07bd80 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.567+0000 7f0c5affd700 1 -- 172.21.15.6:0/1830249930 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c4c0042c0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.567+0000 7f0c5affd700 1 -- 172.21.15.6:0/1830249930 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c011a00 con 0x7f0c5c07bd80 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 -- 172.21.15.6:0/1830249930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 msgr2=0x7f0c5c07a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1830249930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0098a0 tx=0x7f0c4c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 -- 172.21.15.6:0/1830249930 shutdown_connections 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1830249930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c07a280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 -- 172.21.15.6:0/1830249930 >> 172.21.15.6:0/1830249930 conn(0x7f0c5c100fe0 msgr2=0x7f0c5c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 -- 172.21.15.6:0/1830249930 shutdown_connections 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.568+0000 7f0c62a46700 1 -- 172.21.15.6:0/1830249930 wait complete. 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.569+0000 7f0c62a46700 1 Processor -- start 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.569+0000 7f0c62a46700 1 -- start start 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c62a46700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c62a46700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19c030 con 0x7f0c5c07bd80 2023-11-16T23:58:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c5bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33748/0 (socket says 172.21.15.6:33748) 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c5bfff700 1 -- 172.21.15.6:0/1167965616 learned_addr learned my addr 172.21.15.6:0/1167965616 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c5bfff700 1 -- 172.21.15.6:0/1167965616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c009550 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.570+0000 7f0c5bfff700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006af0 tx=0x7f0c4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.571+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c003a40 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.571+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c4c003ba0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.571+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c01a700 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.571+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c10be00 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.571+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c10bc70 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.572+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f0c4c024ad0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.572+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c062120 con 0x7f0c5c07bd80 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.572+0000 7f0c597fa700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 0x7f0c4403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.572+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0c4c04c630 con 0x7f0c5c07bd80 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.573+0000 7f0c5b7fe700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 0x7f0c4403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.573+0000 7f0c5b7fe700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 0x7f0c4403acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006fd0 tx=0x7f0c50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.578+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0c4c018bd0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.786+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0c5c04e8c0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.787+0000 7f0c597fa700 1 -- 172.21.15.6:0/1167965616 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0c4c0187c0 con 0x7f0c5c07bd80 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 msgr2=0x7f0c4403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 0x7f0c4403acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006fd0 tx=0x7f0c50006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 msgr2=0x7f0c5c19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006af0 tx=0x7f0c4c011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 shutdown_connections 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f0c44038820 0x7f0c4403acd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 --2- 172.21.15.6:0/1167965616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c5c07bd80 0x7f0c5c19baf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 >> 172.21.15.6:0/1167965616 conn(0x7f0c5c100fe0 msgr2=0x7f0c5c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 shutdown_connections 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:22.790+0000 7f0c62a46700 1 -- 172.21.15.6:0/1167965616 wait complete. 2023-11-16T23:58:23.346 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2023-11-16T23:58:23.651 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:23 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1167965616' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T23:58:24.548 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/490204367' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-16T23:58:24.548 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setuser ceph since I am not root 2023-11-16T23:58:24.549 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setgroup ceph since I am not root 2023-11-16T23:58:24.900 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:24.547+0000 7f3ce37b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T23:58:24.900 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:24.657+0000 7f3ce37b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T23:58:24.900 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:24.774+0000 7f3ce37b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T23:58:24.901 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:24.898+0000 7f3ce37b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T23:58:25.023 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.933+0000 7f8b38f6b700 1 Processor -- start 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b38f6b700 1 -- start start 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b38f6b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b38f6b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3407a7e0 con 0x7f8b3407bda0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33752/0 (socket says 172.21.15.6:33752) 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.934+0000 7f8b3259c700 1 -- 172.21.15.6:0/2254611031 learned_addr learned my addr 172.21.15.6:0/2254611031 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.935+0000 7f8b3259c700 1 -- 172.21.15.6:0/2254611031 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3407a920 con 0x7f8b3407bda0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.935+0000 7f8b3259c700 1 --2- 172.21.15.6:0/2254611031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b240098a0 tx=0x7f8b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c2a3ac642e36c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.935+0000 7f8b3159a700 1 -- 172.21.15.6:0/2254611031 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b24004160 con 0x7f8b3407bda0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.935+0000 7f8b3159a700 1 -- 172.21.15.6:0/2254611031 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b240042c0 con 0x7f8b3407bda0 2023-11-16T23:58:25.024 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.935+0000 7f8b3159a700 1 -- 172.21.15.6:0/2254611031 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b24011a00 con 0x7f8b3407bda0 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 -- 172.21.15.6:0/2254611031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 msgr2=0x7f8b3407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/2254611031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b240098a0 tx=0x7f8b24009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 -- 172.21.15.6:0/2254611031 shutdown_connections 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/2254611031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3407a2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 -- 172.21.15.6:0/2254611031 >> 172.21.15.6:0/2254611031 conn(0x7f8b34100fe0 msgr2=0x7f8b341033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 -- 172.21.15.6:0/2254611031 shutdown_connections 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.936+0000 7f8b38f6b700 1 -- 172.21.15.6:0/2254611031 wait complete. 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.937+0000 7f8b38f6b700 1 Processor -- start 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.937+0000 7f8b38f6b700 1 -- start start 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.937+0000 7f8b38f6b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.937+0000 7f8b38f6b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3419c030 con 0x7f8b3407bda0 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.938+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:25.025 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.938+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33756/0 (socket says 172.21.15.6:33756) 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.938+0000 7f8b3259c700 1 -- 172.21.15.6:0/490204367 learned_addr learned my addr 172.21.15.6:0/490204367 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.938+0000 7f8b3259c700 1 -- 172.21.15.6:0/490204367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24009550 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.938+0000 7f8b3259c700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b24006af0 tx=0x7f8b24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.939+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b24003a40 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.939+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b24003ba0 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.939+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b2401a700 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.939+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3410be40 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.939+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3410bcb0 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.940+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b34062120 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.940+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f8b24024ad0 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.940+0000 7f8b237fe700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 0x7f8b1c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.940+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8b2404c630 con 0x7f8b3407bda0 2023-11-16T23:58:25.026 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.941+0000 7f8b31d9b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 0x7f8b1c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.941+0000 7f8b31d9b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 0x7f8b1c03ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006fd0 tx=0x7f8b28006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:23.945+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b24018bd0 con 0x7f8b3407bda0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.207+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8b3404e8c0 con 0x7f8b3407bda0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.284+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8b2402d770 con 0x7f8b3407bda0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.286+0000 7f8b237fe700 1 -- 172.21.15.6:0/490204367 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8b240172b0 con 0x7f8b3407bda0 2023-11-16T23:58:25.027 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.288+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 msgr2=0x7f8b1c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:25.058 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.288+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 0x7f8b1c03ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006fd0 tx=0x7f8b28006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.288+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 msgr2=0x7f8b3419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.288+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b24006af0 tx=0x7f8b24011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 shutdown_connections 2023-11-16T23:58:25.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f8b1c0387d0 0x7f8b1c03ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 --2- 172.21.15.6:0/490204367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b3407bda0 0x7f8b3419baf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 >> 172.21.15.6:0/490204367 conn(0x7f8b34100fe0 msgr2=0x7f8b34101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 shutdown_connections 2023-11-16T23:58:25.431 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:24.289+0000 7f8b38f6b700 1 -- 172.21.15.6:0/490204367 wait complete. 2023-11-16T23:58:25.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/490204367' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-16T23:58:25.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-mon[97840]: mgrmap e5: x(active, since 5s) 2023-11-16T23:58:25.432 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:25.153+0000 7f3ce37b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T23:58:25.700 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:25.572+0000 7f3ce37b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T23:58:25.951 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:25.699+0000 7f3ce37b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T23:58:25.951 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:25.868+0000 7f3ce37b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T23:58:26.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:26.003+0000 7f3ce37b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T23:58:26.633 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:26 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2343643030' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.642+0000 7f1519f99700 1 Processor -- start 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.642+0000 7f1519f99700 1 -- start start 2023-11-16T23:58:26.831 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.642+0000 7f1519f99700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.642+0000 7f1519f99700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151407a7e0 con 0x7f151407bda0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.642+0000 7f15137fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15137fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33778/0 (socket says 172.21.15.6:33778) 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15137fe700 1 -- 172.21.15.6:0/4092072851 learned_addr learned my addr 172.21.15.6:0/4092072851 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15137fe700 1 -- 172.21.15.6:0/4092072851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151407a920 con 0x7f151407bda0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15137fe700 1 --2- 172.21.15.6:0/4092072851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15040098a0 tx=0x7f1504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4d246031ebae2b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15127fc700 1 -- 172.21.15.6:0/4092072851 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1504004160 con 0x7f151407bda0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15127fc700 1 -- 172.21.15.6:0/4092072851 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15040042c0 con 0x7f151407bda0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.643+0000 7f15127fc700 1 -- 172.21.15.6:0/4092072851 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1504011a00 con 0x7f151407bda0 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 -- 172.21.15.6:0/4092072851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 msgr2=0x7f151407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:26.832 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 --2- 172.21.15.6:0/4092072851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15040098a0 tx=0x7f1504009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 -- 172.21.15.6:0/4092072851 shutdown_connections 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 --2- 172.21.15.6:0/4092072851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151407a2a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 -- 172.21.15.6:0/4092072851 >> 172.21.15.6:0/4092072851 conn(0x7f1514100fe0 msgr2=0x7f15141033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 -- 172.21.15.6:0/4092072851 shutdown_connections 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.644+0000 7f1519f99700 1 -- 172.21.15.6:0/4092072851 wait complete. 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.645+0000 7f1519f99700 1 Processor -- start 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.645+0000 7f1519f99700 1 -- start start 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f1519f99700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f1519f99700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151419c030 con 0x7f151407bda0 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f15137fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f15137fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33790/0 (socket says 172.21.15.6:33790) 2023-11-16T23:58:26.833 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f15137fe700 1 -- 172.21.15.6:0/2343643030 learned_addr learned my addr 172.21.15.6:0/2343643030 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.646+0000 7f15137fe700 1 -- 172.21.15.6:0/2343643030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009550 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f15137fe700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1504006af0 tx=0x7f1504011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1504003a40 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1504003ba0 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f150401a700 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151410be40 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.647+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151410bcb0 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.648+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f1504024ad0 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.648+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1514062120 con 0x7f151407bda0 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.649+0000 7f1510ff9700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 0x7f14fc03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:26.834 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.649+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f150404c6c0 con 0x7f151407bda0 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.649+0000 7f1512ffd700 1 -- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 msgr2=0x7f14fc03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.649+0000 7f1512ffd700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 0x7f14fc03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.658+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1504024430 con 0x7f151407bda0 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.850+0000 7f1512ffd700 1 -- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 msgr2=0x7f14fc03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.850+0000 7f1512ffd700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 0x7f14fc03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.924+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f151404e8c0 con 0x7f151407bda0 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.925+0000 7f1510ff9700 1 -- 172.21.15.6:0/2343643030 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1504018350 con 0x7f151407bda0 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 msgr2=0x7f14fc03acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 0x7f14fc03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 msgr2=0x7f151419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1504006af0 tx=0x7f1504011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 shutdown_connections 2023-11-16T23:58:26.835 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f14fc038820 0x7f14fc03acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 --2- 172.21.15.6:0/2343643030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f151407bda0 0x7f151419baf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.927+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 >> 172.21.15.6:0/2343643030 conn(0x7f1514100fe0 msgr2=0x7f1514101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.928+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 shutdown_connections 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:25.928+0000 7f1519f99700 1 -- 172.21.15.6:0/2343643030 wait complete. 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2023-11-16T23:58:26.836 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2023-11-16T23:58:26.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:26.568+0000 7f3ce37b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T23:58:27.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:27.369+0000 7f3ce37b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T23:58:27.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:27.586+0000 7f3ce37b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T23:58:28.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:28.566+0000 7f3ce37b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:58:28.922 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:28.691+0000 7f3ce37b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T23:58:29.578 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:28.919+0000 7f3ce37b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:58:29.578 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:29.292+0000 7f3ce37b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T23:58:29.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:29.576+0000 7f3ce37b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T23:58:29.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:29.775+0000 7f3ce37b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T23:58:30.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:30 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:30.129+0000 7f3ce37b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T23:58:31.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:31.090+0000 7f3ce37b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T23:58:31.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:31.269+0000 7f3ce37b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T23:58:31.962 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:31.524+0000 7f3ce37b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T23:58:31.962 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:31.676+0000 7f3ce37b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T23:58:33.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:31.960+0000 7f3ce37b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T23:58:33.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:33.056+0000 7f3ce37b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: Active manager daemon x restarted 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: Activating manager daemon x 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: osdmap e2: 0 total, 0 up, 0 in 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: mgrmap e6: x(active, starting, since 0.00799359s) 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:58:34.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: Manager daemon x is now available 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: Found migration_current of "None". Setting to last migration. 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:34.099 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:33 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.487+0000 7f7d16403700 1 Processor -- start 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.487+0000 7f7d16403700 1 -- start start 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.487+0000 7f7d16403700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:34.628 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.487+0000 7f7d16403700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1007a7c0 con 0x7f7d1007bd80 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.488+0000 7f7d0ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.488+0000 7f7d0ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33792/0 (socket says 172.21.15.6:33792) 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.488+0000 7f7d0ffff700 1 -- 172.21.15.6:0/1212317246 learned_addr learned my addr 172.21.15.6:0/1212317246 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.488+0000 7f7d0ffff700 1 -- 172.21.15.6:0/1212317246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1007a900 con 0x7f7d1007bd80 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.488+0000 7f7d0ffff700 1 --2- 172.21.15.6:0/1212317246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098a0 tx=0x7f7d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a3a424ea4e417 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d0effd700 1 -- 172.21.15.6:0/1212317246 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d00004160 con 0x7f7d1007bd80 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d0effd700 1 -- 172.21.15.6:0/1212317246 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d000042c0 con 0x7f7d1007bd80 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d0effd700 1 -- 172.21.15.6:0/1212317246 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d00011a00 con 0x7f7d1007bd80 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d16403700 1 -- 172.21.15.6:0/1212317246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 msgr2=0x7f7d1007a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d16403700 1 --2- 172.21.15.6:0/1212317246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098a0 tx=0x7f7d00009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d16403700 1 -- 172.21.15.6:0/1212317246 shutdown_connections 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d16403700 1 --2- 172.21.15.6:0/1212317246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1007a280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.629 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.489+0000 7f7d16403700 1 -- 172.21.15.6:0/1212317246 >> 172.21.15.6:0/1212317246 conn(0x7f7d10100fe0 msgr2=0x7f7d101033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.490+0000 7f7d16403700 1 -- 172.21.15.6:0/1212317246 shutdown_connections 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.490+0000 7f7d16403700 1 -- 172.21.15.6:0/1212317246 wait complete. 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d16403700 1 Processor -- start 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d16403700 1 -- start start 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d16403700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d16403700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019c030 con 0x7f7d1007bd80 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d0ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d0ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33808/0 (socket says 172.21.15.6:33808) 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.491+0000 7f7d0ffff700 1 -- 172.21.15.6:0/1525512549 learned_addr learned my addr 172.21.15.6:0/1525512549 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d0ffff700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009550 con 0x7f7d1007bd80 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d0ffff700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006af0 tx=0x7f7d00011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:34.630 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d00003a40 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d00003ba0 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1010be00 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d0001a700 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.492+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1010bc70 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.493+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7d0001a930 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.494+0000 7f7d0d7fa700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.494+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cf803b3e0 con 0x7f7cf8038820 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.494+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7d0004c8a0 con 0x7f7d1007bd80 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.494+0000 7f7d0f7fe700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.494+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.695+0000 7f7d0f7fe700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:34.631 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:27.695+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:28.096+0000 7f7d0f7fe700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:28.096+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:28.897+0000 7f7d0f7fe700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:28.897+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:30.498+0000 7f7d0f7fe700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3715783566 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:30.498+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:33.065+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f7d0002d910 con 0x7f7d1007bd80 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:33.065+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 msgr2=0x7f7cf803acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:33.065+0000 7f7d0d7fa700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.065+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f7d0004e2b0 con 0x7f7d1007bd80 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.065+0000 7f7d0d7fa700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 0x7f7cf803e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.065+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cf803b3e0 con 0x7f7cf803bfb0 2023-11-16T23:58:34.632 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.066+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 0x7f7cf803e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.067+0000 7f7d0f7fe700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 0x7f7cf803e390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003a10 tx=0x7f7d040092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.068+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f7cf803b3e0 con 0x7f7cf803bfb0 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7d1019c170 con 0x7f7cf803bfb0 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d0d7fa700 1 -- 172.21.15.6:0/1525512549 <== mgr.14116 v2:172.21.15.6:6800/610057204 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7d0400a050 con 0x7f7cf803bfb0 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 msgr2=0x7f7cf803e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d16403700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 0x7f7cf803e390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003a10 tx=0x7f7d040092b0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 msgr2=0x7f7d1019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.075+0000 7f7d16403700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006af0 tx=0x7f7d00011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 shutdown_connections 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7cf803bfb0 0x7f7cf803e390 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d1007bd80 0x7f7d1019baf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.633 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 --2- 172.21.15.6:0/1525512549 >> [v2:172.21.15.6:6800/3715783566,v1:172.21.15.6:6801/3715783566] conn(0x7f7cf8038820 0x7f7cf803acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:34.634 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 >> 172.21.15.6:0/1525512549 conn(0x7f7d10100fe0 msgr2=0x7f7d10101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:34.634 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 shutdown_connections 2023-11-16T23:58:34.634 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:34.076+0000 7f7d16403700 1 -- 172.21.15.6:0/1525512549 wait complete. 2023-11-16T23:58:34.634 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2023-11-16T23:58:34.634 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2023-11-16T23:58:35.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:35 smithi006 ceph-mon[97840]: mgrmap e7: x(active, since 1.00878s) 2023-11-16T23:58:35.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:35 smithi006 ceph-mon[97840]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-16T23:58:35.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:35 smithi006 ceph-mon[97840]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.282+0000 7f4359127700 1 Processor -- start 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.282+0000 7f4359127700 1 -- start start 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4359127700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4359127700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43541062a0 con 0x7f4354105950 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4352d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4352d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52366/0 (socket says 172.21.15.6:52366) 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4352d9d700 1 -- 172.21.15.6:0/2370308045 learned_addr learned my addr 172.21.15.6:0/2370308045 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.283+0000 7f4352d9d700 1 -- 172.21.15.6:0/2370308045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43541063e0 con 0x7f4354105950 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4352d9d700 1 --2- 172.21.15.6:0/2370308045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f43440098a0 tx=0x7f4344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e909db6fe8277f8d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:36.112 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4351d9b700 1 -- 172.21.15.6:0/2370308045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4344004160 con 0x7f4354105950 2023-11-16T23:58:36.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4351d9b700 1 -- 172.21.15.6:0/2370308045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43440042c0 con 0x7f4354105950 2023-11-16T23:58:36.113 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4351d9b700 1 -- 172.21.15.6:0/2370308045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4344011a00 con 0x7f4354105950 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4359127700 1 -- 172.21.15.6:0/2370308045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 msgr2=0x7f4354105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.284+0000 7f4359127700 1 --2- 172.21.15.6:0/2370308045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f43440098a0 tx=0x7f4344009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.285+0000 7f4359127700 1 -- 172.21.15.6:0/2370308045 shutdown_connections 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.285+0000 7f4359127700 1 --2- 172.21.15.6:0/2370308045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354105d60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.285+0000 7f4359127700 1 -- 172.21.15.6:0/2370308045 >> 172.21.15.6:0/2370308045 conn(0x7f4354100fe0 msgr2=0x7f43541033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:36.163 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.285+0000 7f4359127700 1 -- 172.21.15.6:0/2370308045 shutdown_connections 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.285+0000 7f4359127700 1 -- 172.21.15.6:0/2370308045 wait complete. 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4359127700 1 Processor -- start 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4359127700 1 -- start start 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4359127700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4359127700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354199dc0 con 0x7f4354105950 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4352d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4352d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52376/0 (socket says 172.21.15.6:52376) 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.286+0000 7f4352d9d700 1 -- 172.21.15.6:0/1668355043 learned_addr learned my addr 172.21.15.6:0/1668355043 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:36.164 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f4352d9d700 1 -- 172.21.15.6:0/1668355043 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344009550 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f4352d9d700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4344009fb0 tx=0x7f4344011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4344003930 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4344003a90 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43541079a0 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4344019610 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.287+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4354107810 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.288+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4344019880 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.288+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4354062120 con 0x7f4354105950 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.289+0000 7f433bfff700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 0x7f433c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:36.165 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.289+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f434404cae0 con 0x7f4354105950 2023-11-16T23:58:36.166 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.289+0000 7f435259c700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 0x7f433c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:36.166 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.290+0000 7f435259c700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 0x7f433c03ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4348006fd0 tx=0x7f4348006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:36.166 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.295+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f434402b340 con 0x7f4354105950 2023-11-16T23:58:36.166 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.516+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f43541023b0 con 0x7f433c0387d0 2023-11-16T23:58:36.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.524+0000 7f433bfff700 1 -- 172.21.15.6:0/1668355043 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f43541023b0 con 0x7f433c0387d0 2023-11-16T23:58:36.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 msgr2=0x7f433c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:36.168 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 0x7f433c03ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4348006fd0 tx=0x7f4348006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:36.184 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 msgr2=0x7f4354199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:36.184 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4344009fb0 tx=0x7f4344011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:36.184 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 shutdown_connections 2023-11-16T23:58:36.184 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f433c0387d0 0x7f433c03ac80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:36.184 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 --2- 172.21.15.6:0/1668355043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4354105950 0x7f4354199880 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:36.185 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.527+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 >> 172.21.15.6:0/1668355043 conn(0x7f4354100fe0 msgr2=0x7f4354101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:36.185 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.528+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 shutdown_connections 2023-11-16T23:58:36.185 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:35.528+0000 7f4359127700 1 -- 172.21.15.6:0/1668355043 wait complete. 2023-11-16T23:58:36.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: [16/Nov/2023:23:58:34] ENGINE Bus STARTING 2023-11-16T23:58:36.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: [16/Nov/2023:23:58:35] ENGINE Serving on http://172.21.15.6:8765 2023-11-16T23:58:36.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: mgrmap e8: x(active, since 2s) 2023-11-16T23:58:36.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: [16/Nov/2023:23:58:35] ENGINE Serving on https://172.21.15.6:7150 2023-11-16T23:58:36.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: [16/Nov/2023:23:58:35] ENGINE Bus STARTED 2023-11-16T23:58:36.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:36.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:36.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:36 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:37.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:37 smithi006 ceph-mon[97840]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.813+0000 7efc04f31700 1 Processor -- start 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efc04f31700 1 -- start start 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efc04f31700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efc04f31700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0007a7e0 con 0x7efc0007bda0 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efbfe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:37.591 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efbfe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52382/0 (socket says 172.21.15.6:52382) 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.814+0000 7efbfe59c700 1 -- 172.21.15.6:0/2550063702 learned_addr learned my addr 172.21.15.6:0/2550063702 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.815+0000 7efbfe59c700 1 -- 172.21.15.6:0/2550063702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0007a920 con 0x7efc0007bda0 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.815+0000 7efbfe59c700 1 --2- 172.21.15.6:0/2550063702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efbf00098a0 tx=0x7efbf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b2b342976426d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.815+0000 7efbfd59a700 1 -- 172.21.15.6:0/2550063702 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf0004160 con 0x7efc0007bda0 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.815+0000 7efbfd59a700 1 -- 172.21.15.6:0/2550063702 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efbf00042c0 con 0x7efc0007bda0 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.815+0000 7efbfd59a700 1 -- 172.21.15.6:0/2550063702 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf0011a00 con 0x7efc0007bda0 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 -- 172.21.15.6:0/2550063702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 msgr2=0x7efc0007a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 --2- 172.21.15.6:0/2550063702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efbf00098a0 tx=0x7efbf0009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 -- 172.21.15.6:0/2550063702 shutdown_connections 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 --2- 172.21.15.6:0/2550063702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0007a2a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 -- 172.21.15.6:0/2550063702 >> 172.21.15.6:0/2550063702 conn(0x7efc00100fe0 msgr2=0x7efc001033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 -- 172.21.15.6:0/2550063702 shutdown_connections 2023-11-16T23:58:37.592 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.816+0000 7efc04f31700 1 -- 172.21.15.6:0/2550063702 wait complete. 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.817+0000 7efc04f31700 1 Processor -- start 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.817+0000 7efc04f31700 1 -- start start 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efc04f31700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efc04f31700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0019c030 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efbfe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efbfe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52394/0 (socket says 172.21.15.6:52394) 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efbfe59c700 1 -- 172.21.15.6:0/3508853570 learned_addr learned my addr 172.21.15.6:0/3508853570 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.818+0000 7efbfe59c700 1 -- 172.21.15.6:0/3508853570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf0009550 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efbfe59c700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006af0 tx=0x7efbf0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf0003a40 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efbf0003ba0 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0010be40 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efbf001a700 con 0x7efc0007bda0 2023-11-16T23:58:37.593 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.819+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0010bcb0 con 0x7efc0007bda0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.820+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7efbf0024430 con 0x7efc0007bda0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.820+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc00062120 con 0x7efc0007bda0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.820+0000 7efbef7fe700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 0x7efbe803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.820+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efbf004c7a0 con 0x7efc0007bda0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.821+0000 7efbfdd9b700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 0x7efbe803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.821+0000 7efbfdd9b700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 0x7efbe803ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006fd0 tx=0x7efbf4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:36.826+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efbf00189a0 con 0x7efc0007bda0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.033+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7efc00102520 con 0x7efbe80387d0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.034+0000 7efbef7fe700 1 -- 172.21.15.6:0/3508853570 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7efc00102520 con 0x7efbe80387d0 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 msgr2=0x7efbe803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 0x7efbe803ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006fd0 tx=0x7efbf4006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:37.594 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 msgr2=0x7efc0019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf0006af0 tx=0x7efbf0011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 shutdown_connections 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7efbe80387d0 0x7efbe803ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.036+0000 7efc04f31700 1 --2- 172.21.15.6:0/3508853570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc0007bda0 0x7efc0019baf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.037+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 >> 172.21.15.6:0/3508853570 conn(0x7efc00100fe0 msgr2=0x7efc00101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.037+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 shutdown_connections 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:37.037+0000 7efc04f31700 1 -- 172.21.15.6:0/3508853570 wait complete. 2023-11-16T23:58:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2023-11-16T23:58:38.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-mon[97840]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: Generating public/private rsa key pair. 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: Your identification has been saved in /tmp/tmpc9xqxzpk/key. 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: Your public key has been saved in /tmp/tmpc9xqxzpk/key.pub. 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: The key fingerprint is: 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: SHA256:+4rqzKRyjhcKWDwIQeEUWI38MeIQT1/ohSiqbm/uS5k ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:39.072 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: The key's randomart image is: 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: +---[RSA 3072]----+ 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |=X=+ o. | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |Bo=.*.. | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |oB.+.+ | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |o = o | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |o. . S | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |+ . o . | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |o. E. . | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |o++* . . | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: |o=B**.. ... | 2023-11-16T23:58:39.073 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: +----[SHA256]-----+ 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.162+0000 7f132bb15700 1 Processor -- start 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.162+0000 7f132bb15700 1 -- start start 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.162+0000 7f132bb15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.162+0000 7f132bb15700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132407a800 con 0x7f132407bdc0 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13298b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13298b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52406/0 (socket says 172.21.15.6:52406) 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13298b1700 1 -- 172.21.15.6:0/2638414313 learned_addr learned my addr 172.21.15.6:0/2638414313 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13298b1700 1 -- 172.21.15.6:0/2638414313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132407a940 con 0x7f132407bdc0 2023-11-16T23:58:39.379 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13298b1700 1 --2- 172.21.15.6:0/2638414313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13200098a0 tx=0x7f1320009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94533e9a791b8bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13288af700 1 -- 172.21.15.6:0/2638414313 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1320004160 con 0x7f132407bdc0 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13288af700 1 -- 172.21.15.6:0/2638414313 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13200042c0 con 0x7f132407bdc0 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.163+0000 7f13288af700 1 -- 172.21.15.6:0/2638414313 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1320011a00 con 0x7f132407bdc0 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 -- 172.21.15.6:0/2638414313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 msgr2=0x7f132407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 --2- 172.21.15.6:0/2638414313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13200098a0 tx=0x7f1320009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 -- 172.21.15.6:0/2638414313 shutdown_connections 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 --2- 172.21.15.6:0/2638414313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132407a2c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 -- 172.21.15.6:0/2638414313 >> 172.21.15.6:0/2638414313 conn(0x7f1324100fe0 msgr2=0x7f13241033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 -- 172.21.15.6:0/2638414313 shutdown_connections 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.164+0000 7f132bb15700 1 -- 172.21.15.6:0/2638414313 wait complete. 2023-11-16T23:58:39.380 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.165+0000 7f132bb15700 1 Processor -- start 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.165+0000 7f132bb15700 1 -- start start 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f132bb15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f132bb15700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132419c060 con 0x7f132407bdc0 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f13298b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f13298b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52422/0 (socket says 172.21.15.6:52422) 2023-11-16T23:58:39.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f13298b1700 1 -- 172.21.15.6:0/3847294863 learned_addr learned my addr 172.21.15.6:0/3847294863 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.166+0000 7f13298b1700 1 -- 172.21.15.6:0/3847294863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009550 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f13298b1700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1320006b20 tx=0x7f1320011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1320003b70 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1320003cd0 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132410be20 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13200247c0 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.167+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132410bc90 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.168+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1324062120 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.168+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f132001ab40 con 0x7f132407bdc0 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.169+0000 7f131affd700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 0x7f131003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:39.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.169+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f132004c9c0 con 0x7f132407bdc0 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.169+0000 7f13290b0700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 0x7f131003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.169+0000 7f13290b0700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 0x7f131003acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1314006fd0 tx=0x7f1314006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.174+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f132001a950 con 0x7f132407bdc0 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.382+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1324102560 con 0x7f1310038820 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.787+0000 7f131affd700 1 -- 172.21.15.6:0/3847294863 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1324102560 con 0x7f1310038820 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 msgr2=0x7f131003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:39.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 0x7f131003acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1314006fd0 tx=0x7f1314006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 msgr2=0x7f132419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1320006b20 tx=0x7f1320011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 shutdown_connections 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1310038820 0x7f131003acd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 --2- 172.21.15.6:0/3847294863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f132407bdc0 0x7f132419bb20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 >> 172.21.15.6:0/3847294863 conn(0x7f1324100fe0 msgr2=0x7f1324101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.789+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 shutdown_connections 2023-11-16T23:58:39.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:38.790+0000 7f132bb15700 1 -- 172.21.15.6:0/3847294863 wait complete. 2023-11-16T23:58:39.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:39 smithi006 ceph-mon[97840]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:39.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:39 smithi006 ceph-mon[97840]: Generating ssh key... 2023-11-16T23:58:39.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:39 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:39.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:39 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:40.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:40 smithi006 ceph-mon[97840]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:40.861 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f719b764700 1 Processor -- start 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f719b764700 1 -- start start 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f719b764700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f719b764700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719407a7e0 con 0x7f719407bda0 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f7199500700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f7199500700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41318/0 (socket says 172.21.15.6:41318) 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f7199500700 1 -- 172.21.15.6:0/667814867 learned_addr learned my addr 172.21.15.6:0/667814867 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f7199500700 1 -- 172.21.15.6:0/667814867 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719407a920 con 0x7f719407bda0 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.031+0000 7f7199500700 1 --2- 172.21.15.6:0/667814867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71900098a0 tx=0x7f7190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897854c6d02c6d6e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f718bfff700 1 -- 172.21.15.6:0/667814867 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7190004160 con 0x7f719407bda0 2023-11-16T23:58:40.862 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f718bfff700 1 -- 172.21.15.6:0/667814867 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f71900042c0 con 0x7f719407bda0 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f718bfff700 1 -- 172.21.15.6:0/667814867 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7190011a00 con 0x7f719407bda0 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 -- 172.21.15.6:0/667814867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 msgr2=0x7f719407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 --2- 172.21.15.6:0/667814867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71900098a0 tx=0x7f7190009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 -- 172.21.15.6:0/667814867 shutdown_connections 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 --2- 172.21.15.6:0/667814867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719407a2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 -- 172.21.15.6:0/667814867 >> 172.21.15.6:0/667814867 conn(0x7f7194100fe0 msgr2=0x7f71941033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 -- 172.21.15.6:0/667814867 shutdown_connections 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.032+0000 7f719b764700 1 -- 172.21.15.6:0/667814867 wait complete. 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.033+0000 7f719b764700 1 Processor -- start 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f719b764700 1 -- start start 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f719b764700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f719b764700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719419c030 con 0x7f719407bda0 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f7199500700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:40.863 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f7199500700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41334/0 (socket says 172.21.15.6:41334) 2023-11-16T23:58:40.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.034+0000 7f7199500700 1 -- 172.21.15.6:0/2868230474 learned_addr learned my addr 172.21.15.6:0/2868230474 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:40.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f7199500700 1 -- 172.21.15.6:0/2868230474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009550 con 0x7f719407bda0 2023-11-16T23:58:40.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f7199500700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7190006af0 tx=0x7f7190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:40.864 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7190003a40 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7190003ba0 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f719001a700 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719410be40 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.035+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719410bcb0 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.036+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7190024ad0 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.036+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7194062120 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.037+0000 7f718a7fc700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 0x7f718003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.037+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f719004c6c0 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.037+0000 7f7198cff700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 0x7f718003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.038+0000 7f7198cff700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 0x7f718003acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7184006fd0 tx=0x7f7184006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.042+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7190018b80 con 0x7f719407bda0 2023-11-16T23:58:40.865 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.250+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7194102520 con 0x7f7180038820 2023-11-16T23:58:40.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.251+0000 7f718a7fc700 1 -- 172.21.15.6:0/2868230474 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7194102520 con 0x7f7180038820 2023-11-16T23:58:40.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.253+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 msgr2=0x7f718003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:40.866 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.253+0000 7f719b764700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 0x7f718003acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7184006fd0 tx=0x7f7184006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:40.867 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.253+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 msgr2=0x7f719419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.253+0000 7f719b764700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7190006af0 tx=0x7f7190011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 shutdown_connections 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f7180038820 0x7f718003acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 --2- 172.21.15.6:0/2868230474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f719407bda0 0x7f719419baf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 >> 172.21.15.6:0/2868230474 conn(0x7f7194100fe0 msgr2=0x7f7194101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 shutdown_connections 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:40.254+0000 7f719b764700 1 -- 172.21.15.6:0/2868230474 wait complete. 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2023-11-16T23:58:40.868 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2023-11-16T23:58:42.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:42 smithi006 ceph-mon[97840]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:44.080 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:43 smithi006 ceph-mon[97840]: Deploying cephadm binary to smithi006 2023-11-16T23:58:45.351 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.459+0000 7f1b2bf66700 1 Processor -- start 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.459+0000 7f1b2bf66700 1 -- start start 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b2bf66700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b2bf66700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2407a800 con 0x7f1b2407bdc0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b29d02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b29d02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41344/0 (socket says 172.21.15.6:41344) 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b29d02700 1 -- 172.21.15.6:0/646914071 learned_addr learned my addr 172.21.15.6:0/646914071 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.460+0000 7f1b29d02700 1 -- 172.21.15.6:0/646914071 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2407a940 con 0x7f1b2407bdc0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b29d02700 1 --2- 172.21.15.6:0/646914071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b200098a0 tx=0x7f1b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90865cf288a1c01 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b28d00700 1 -- 172.21.15.6:0/646914071 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b20004160 con 0x7f1b2407bdc0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b28d00700 1 -- 172.21.15.6:0/646914071 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b200042c0 con 0x7f1b2407bdc0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b28d00700 1 -- 172.21.15.6:0/646914071 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b20011a00 con 0x7f1b2407bdc0 2023-11-16T23:58:45.352 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b2bf66700 1 -- 172.21.15.6:0/646914071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 msgr2=0x7f1b2407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.461+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/646914071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b200098a0 tx=0x7f1b20009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.462+0000 7f1b2bf66700 1 -- 172.21.15.6:0/646914071 shutdown_connections 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.462+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/646914071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2407a2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.462+0000 7f1b2bf66700 1 -- 172.21.15.6:0/646914071 >> 172.21.15.6:0/646914071 conn(0x7f1b24100fe0 msgr2=0x7f1b241033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.462+0000 7f1b2bf66700 1 -- 172.21.15.6:0/646914071 shutdown_connections 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.462+0000 7f1b2bf66700 1 -- 172.21.15.6:0/646914071 wait complete. 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.463+0000 7f1b2bf66700 1 Processor -- start 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.463+0000 7f1b2bf66700 1 -- start start 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.464+0000 7f1b2bf66700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.464+0000 7f1b2bf66700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2419c060 con 0x7f1b2407bdc0 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.464+0000 7f1b29d02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.464+0000 7f1b29d02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41348/0 (socket says 172.21.15.6:41348) 2023-11-16T23:58:45.353 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.464+0000 7f1b29d02700 1 -- 172.21.15.6:0/3734038256 learned_addr learned my addr 172.21.15.6:0/3734038256 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b29d02700 1 -- 172.21.15.6:0/3734038256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20009550 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b29d02700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006b20 tx=0x7f1b20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b20003b70 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b20003cd0 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b200247c0 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2410be20 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.465+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2410bc90 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.466+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f1b2001a430 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.466+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24062120 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.467+0000 7f1b1affd700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 0x7f1b1003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.467+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b2004c970 con 0x7f1b2407bdc0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.467+0000 7f1b29501700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 0x7f1b1003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:45.354 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.467+0000 7f1b29501700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 0x7f1b1003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006fd0 tx=0x7f1b14006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.471+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b2001a6e0 con 0x7f1b2407bdc0 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:41.683+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b24102560 con 0x7f1b10038820 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.780+0000 7f1b1affd700 1 -- 172.21.15.6:0/3734038256 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f1b24102560 con 0x7f1b10038820 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 msgr2=0x7f1b1003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 0x7f1b1003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006fd0 tx=0x7f1b14006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 msgr2=0x7f1b2419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006b20 tx=0x7f1b20011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 shutdown_connections 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f1b10038820 0x7f1b1003acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 --2- 172.21.15.6:0/3734038256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b2407bdc0 0x7f1b2419bb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 >> 172.21.15.6:0/3734038256 conn(0x7f1b24100fe0 msgr2=0x7f1b24101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 shutdown_connections 2023-11-16T23:58:45.355 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:44.783+0000 7f1b2bf66700 1 -- 172.21.15.6:0/3734038256 wait complete. 2023-11-16T23:58:45.356 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mon service... 2023-11-16T23:58:45.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:45 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:45.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:45 smithi006 ceph-mon[97840]: Added host smithi006 2023-11-16T23:58:45.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:45 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.960+0000 7f3d82a0a700 1 Processor -- start 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.960+0000 7f3d82a0a700 1 -- start start 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d82a0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d82a0a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c07a7e0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:46.780 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41358/0 (socket says 172.21.15.6:41358) 2023-11-16T23:58:46.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2277441894 learned_addr learned my addr 172.21.15.6:0/2277441894 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:46.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.961+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2277441894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c07a920 con 0x7f3d7c07bda0 2023-11-16T23:58:46.781 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2277441894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0098a0 tx=0x7f3d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf3cc944a2c842c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:46.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d7affd700 1 -- 172.21.15.6:0/2277441894 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c004160 con 0x7f3d7c07bda0 2023-11-16T23:58:46.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d7affd700 1 -- 172.21.15.6:0/2277441894 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d6c0042c0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.782 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d7affd700 1 -- 172.21.15.6:0/2277441894 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c011a00 con 0x7f3d7c07bda0 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d82a0a700 1 -- 172.21.15.6:0/2277441894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 msgr2=0x7f3d7c07a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.962+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/2277441894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0098a0 tx=0x7f3d6c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.963+0000 7f3d82a0a700 1 -- 172.21.15.6:0/2277441894 shutdown_connections 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.963+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/2277441894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c07a2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.963+0000 7f3d82a0a700 1 -- 172.21.15.6:0/2277441894 >> 172.21.15.6:0/2277441894 conn(0x7f3d7c100fe0 msgr2=0x7f3d7c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.963+0000 7f3d82a0a700 1 -- 172.21.15.6:0/2277441894 shutdown_connections 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.963+0000 7f3d82a0a700 1 -- 172.21.15.6:0/2277441894 wait complete. 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d82a0a700 1 Processor -- start 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d82a0a700 1 -- start start 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d82a0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d82a0a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c19c030 con 0x7f3d7c07bda0 2023-11-16T23:58:46.783 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41372/0 (socket says 172.21.15.6:41372) 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.964+0000 7f3d7bfff700 1 -- 172.21.15.6:0/3784417834 learned_addr learned my addr 172.21.15.6:0/3784417834 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d7bfff700 1 -- 172.21.15.6:0/3784417834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c009550 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006af0 tx=0x7f3d6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c003a40 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d6c003ba0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c01a700 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c10be40 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.965+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c10bcb0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.966+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c062120 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.966+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3d6c024ad0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.784 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.967+0000 7f3d797fa700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 0x7f3d6403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.967+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d6c04c6c0 con 0x7f3d7c07bda0 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.967+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 0x7f3d6403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.968+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 0x7f3d6403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d70006fd0 tx=0x7f3d70006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:45.972+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3d6c018b80 con 0x7f3d7c07bda0 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.194+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7c102520 con 0x7f3d64038820 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.200+0000 7f3d797fa700 1 -- 172.21.15.6:0/3784417834 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3d7c102520 con 0x7f3d64038820 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 msgr2=0x7f3d6403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 0x7f3d6403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d70006fd0 tx=0x7f3d70006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 msgr2=0x7f3d7c19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006af0 tx=0x7f3d6c011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:46.785 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 shutdown_connections 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f3d64038820 0x7f3d6403acd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 --2- 172.21.15.6:0/3784417834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c07bda0 0x7f3d7c19baf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 >> 172.21.15.6:0/3784417834 conn(0x7f3d7c100fe0 msgr2=0x7f3d7c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 shutdown_connections 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:46.202+0000 7f3d82a0a700 1 -- 172.21.15.6:0/3784417834 wait complete. 2023-11-16T23:58:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mgr service... 2023-11-16T23:58:47.319 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:47 smithi006 ceph-mon[97840]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:47.319 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:47 smithi006 ceph-mon[97840]: Saving service mon spec with placement count:5 2023-11-16T23:58:47.319 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:47 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:48.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-16T23:58:48.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.622+0000 7f50bcfbb700 1 Processor -- start 2023-11-16T23:58:48.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50bcfbb700 1 -- start start 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50bcfbb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50bcfbb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b807a820 con 0x7f50b807bde0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41376/0 (socket says 172.21.15.6:41376) 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.623+0000 7f50b659c700 1 -- 172.21.15.6:0/4178453139 learned_addr learned my addr 172.21.15.6:0/4178453139 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.624+0000 7f50b659c700 1 -- 172.21.15.6:0/4178453139 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b807a960 con 0x7f50b807bde0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.624+0000 7f50b659c700 1 --2- 172.21.15.6:0/4178453139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50a80098a0 tx=0x7f50a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cee188b13649548 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.624+0000 7f50b559a700 1 -- 172.21.15.6:0/4178453139 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50a8004160 con 0x7f50b807bde0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.624+0000 7f50b559a700 1 -- 172.21.15.6:0/4178453139 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50a80042c0 con 0x7f50b807bde0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.624+0000 7f50b559a700 1 -- 172.21.15.6:0/4178453139 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50a8011a00 con 0x7f50b807bde0 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 -- 172.21.15.6:0/4178453139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 msgr2=0x7f50b807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:48.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/4178453139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50a80098a0 tx=0x7f50a8009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 -- 172.21.15.6:0/4178453139 shutdown_connections 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/4178453139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b807a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 -- 172.21.15.6:0/4178453139 >> 172.21.15.6:0/4178453139 conn(0x7f50b8100fe0 msgr2=0x7f50b81033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 -- 172.21.15.6:0/4178453139 shutdown_connections 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.625+0000 7f50bcfbb700 1 -- 172.21.15.6:0/4178453139 wait complete. 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.626+0000 7f50bcfbb700 1 Processor -- start 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.626+0000 7f50bcfbb700 1 -- start start 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.626+0000 7f50bcfbb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.626+0000 7f50bcfbb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b819c050 con 0x7f50b807bde0 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41388/0 (socket says 172.21.15.6:41388) 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50b659c700 1 -- 172.21.15.6:0/2422425696 learned_addr learned my addr 172.21.15.6:0/2422425696 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:48.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50b659c700 1 -- 172.21.15.6:0/2422425696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8009550 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50b659c700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006af0 tx=0x7f50a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.627+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50a8003a40 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.628+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50a8003ba0 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.628+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50a801a700 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.628+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b810be00 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.628+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b810bc70 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.629+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f50a8024430 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.629+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b8195960 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.629+0000 7f50a77fe700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 0x7f50a003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.629+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f50a804c6e0 con 0x7f50b807bde0 2023-11-16T23:58:48.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.629+0000 7f50b5d9b700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 0x7f50a003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.630+0000 7f50b5d9b700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 0x7f50a003ac80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.634+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f50a80189a0 con 0x7f50b807bde0 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.851+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f50b819c6a0 con 0x7f50a00387d0 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.856+0000 7f50a77fe700 1 -- 172.21.15.6:0/2422425696 <== mgr.14116 v2:172.21.15.6:6800/610057204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f50b819c6a0 con 0x7f50a00387d0 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 msgr2=0x7f50a003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 0x7f50a003ac80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 msgr2=0x7f50b819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006af0 tx=0x7f50a8011ad0 comp rx=0 tx=0).stop 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 shutdown_connections 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f50a00387d0 0x7f50a003ac80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 --2- 172.21.15.6:0/2422425696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50b807bde0 0x7f50b819bb10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.858+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 >> 172.21.15.6:0/2422425696 conn(0x7f50b8100fe0 msgr2=0x7f50b8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.859+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 shutdown_connections 2023-11-16T23:58:48.416 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:47.859+0000 7f50bcfbb700 1 -- 172.21.15.6:0/2422425696 wait complete. 2023-11-16T23:58:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:48 smithi006 ceph-mon[97840]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:58:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:48 smithi006 ceph-mon[97840]: Saving service mgr spec with placement count:2 2023-11-16T23:58:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:48 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:48 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:48 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:49.905 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.036+0000 7fdd2c0e0700 1 Processor -- start 2023-11-16T23:58:49.905 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.036+0000 7fdd2c0e0700 1 -- start start 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.036+0000 7fdd2c0e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.036+0000 7fdd2c0e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2407a7e0 con 0x7fdd2407bda0 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.037+0000 7fdd29e7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.037+0000 7fdd29e7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60596/0 (socket says 172.21.15.6:60596) 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.037+0000 7fdd29e7c700 1 -- 172.21.15.6:0/2645909436 learned_addr learned my addr 172.21.15.6:0/2645909436 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.037+0000 7fdd29e7c700 1 -- 172.21.15.6:0/2645909436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2407a920 con 0x7fdd2407bda0 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.037+0000 7fdd29e7c700 1 --2- 172.21.15.6:0/2645909436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd200098a0 tx=0x7fdd20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f834b906bd450f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:49.906 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd28e7a700 1 -- 172.21.15.6:0/2645909436 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd20004160 con 0x7fdd2407bda0 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd28e7a700 1 -- 172.21.15.6:0/2645909436 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd200042c0 con 0x7fdd2407bda0 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd28e7a700 1 -- 172.21.15.6:0/2645909436 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd200119b0 con 0x7fdd2407bda0 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/2645909436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 msgr2=0x7fdd2407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/2645909436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd200098a0 tx=0x7fdd20009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/2645909436 shutdown_connections 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/2645909436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2407a2a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.038+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/2645909436 >> 172.21.15.6:0/2645909436 conn(0x7fdd24100fe0 msgr2=0x7fdd241033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.039+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/2645909436 shutdown_connections 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.039+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/2645909436 wait complete. 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd2c0e0700 1 Processor -- start 2023-11-16T23:58:49.907 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd2c0e0700 1 -- start start 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd2c0e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd2c0e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2419c060 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd29e7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd29e7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60610/0 (socket says 172.21.15.6:60610) 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.040+0000 7fdd29e7c700 1 -- 172.21.15.6:0/3384434462 learned_addr learned my addr 172.21.15.6:0/3384434462 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.041+0000 7fdd29e7c700 1 -- 172.21.15.6:0/3384434462 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009550 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.041+0000 7fdd29e7c700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd20000c00 tx=0x7fdd20011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.041+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd200039f0 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.041+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd20003b50 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.041+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2410be90 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.042+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2410bd00 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.042+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd2001a780 con 0x7fdd2407bda0 2023-11-16T23:58:49.908 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.043+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fdd2001aa40 con 0x7fdd2407bda0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.043+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2410c020 con 0x7fdd2407bda0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.043+0000 7fdd1affd700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 0x7fdd1003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.043+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd2004c7f0 con 0x7fdd2407bda0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.043+0000 7fdd2967b700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 0x7fdd1003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.044+0000 7fdd2967b700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 0x7fdd1003ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006fd0 tx=0x7fdd14006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.049+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdd200237b0 con 0x7fdd2407bda0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.257+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fdd2419c620 con 0x7fdd2407bda0 2023-11-16T23:58:49.909 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.263+0000 7fdd1affd700 1 -- 172.21.15.6:0/3384434462 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fdd2002c350 con 0x7fdd2407bda0 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 msgr2=0x7fdd1003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 0x7fdd1003ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006fd0 tx=0x7fdd14006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 msgr2=0x7fdd2419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd20000c00 tx=0x7fdd20011a30 comp rx=0 tx=0).stop 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 shutdown_connections 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fdd100387d0 0x7fdd1003ac80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:49.910 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 --2- 172.21.15.6:0/3384434462 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd2407bda0 0x7fdd2419bb20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:49.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 >> 172.21.15.6:0/3384434462 conn(0x7fdd24100fe0 msgr2=0x7fdd24101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:49.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 shutdown_connections 2023-11-16T23:58:49.911 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:49.266+0000 7fdd2c0e0700 1 -- 172.21.15.6:0/3384434462 wait complete. 2023-11-16T23:58:50.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:50 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3384434462' entity='client.admin' 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.528+0000 7f8d32376700 1 Processor -- start 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d32376700 1 -- start start 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d32376700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d32376700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c07a820 con 0x7f8d2c07bde0 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60624/0 (socket says 172.21.15.6:60624) 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.529+0000 7f8d2bfff700 1 -- 172.21.15.6:0/987814829 learned_addr learned my addr 172.21.15.6:0/987814829 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.530+0000 7f8d2bfff700 1 -- 172.21.15.6:0/987814829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c07a960 con 0x7f8d2c07bde0 2023-11-16T23:58:51.489 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.530+0000 7f8d2bfff700 1 --2- 172.21.15.6:0/987814829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0098a0 tx=0x7f8d1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9549ba47e1a7646 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.530+0000 7f8d2affd700 1 -- 172.21.15.6:0/987814829 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1c004160 con 0x7f8d2c07bde0 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.530+0000 7f8d2affd700 1 -- 172.21.15.6:0/987814829 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c0042c0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.530+0000 7f8d2affd700 1 -- 172.21.15.6:0/987814829 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1c0119b0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 -- 172.21.15.6:0/987814829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 msgr2=0x7f8d2c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 --2- 172.21.15.6:0/987814829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0098a0 tx=0x7f8d1c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 -- 172.21.15.6:0/987814829 shutdown_connections 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 --2- 172.21.15.6:0/987814829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c07a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 -- 172.21.15.6:0/987814829 >> 172.21.15.6:0/987814829 conn(0x7f8d2c100fe0 msgr2=0x7f8d2c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 -- 172.21.15.6:0/987814829 shutdown_connections 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.531+0000 7f8d32376700 1 -- 172.21.15.6:0/987814829 wait complete. 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d32376700 1 Processor -- start 2023-11-16T23:58:51.490 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d32376700 1 -- start start 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d32376700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d32376700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c19c060 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60628/0 (socket says 172.21.15.6:60628) 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.532+0000 7f8d2bfff700 1 -- 172.21.15.6:0/3425516143 learned_addr learned my addr 172.21.15.6:0/3425516143 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d2bfff700 1 -- 172.21.15.6:0/3425516143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c009550 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d2bfff700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000c00 tx=0x7f8d1c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1c003ac0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c003c20 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c10be50 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.533+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1c01a720 con 0x7f8d2c07bde0 2023-11-16T23:58:51.491 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.534+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c10bcc0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.535+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c10bfe0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.535+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8d1c024440 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.535+0000 7f8d297fa700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 0x7f8d1403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.535+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d1c04c930 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.536+0000 7f8d2b7fe700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 0x7f8d1403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.536+0000 7f8d2b7fe700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 0x7f8d1403ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d20006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.540+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8d1c0246f0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.754+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f8d2c04e8f0 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.760+0000 7f8d297fa700 1 -- 172.21.15.6:0/3425516143 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f8d1c00be80 con 0x7f8d2c07bde0 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 msgr2=0x7f8d1403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 0x7f8d1403ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d20006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:51.492 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 msgr2=0x7f8d2c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000c00 tx=0x7f8d1c011a80 comp rx=0 tx=0).stop 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 shutdown_connections 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f8d140387d0 0x7f8d1403ac80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 --2- 172.21.15.6:0/3425516143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d2c07bde0 0x7f8d2c19bb20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 >> 172.21.15.6:0/3425516143 conn(0x7f8d2c100fe0 msgr2=0x7f8d2c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 shutdown_connections 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:50.763+0000 7f8d32376700 1 -- 172.21.15.6:0/3425516143 wait complete. 2023-11-16T23:58:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2023-11-16T23:58:52.202 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:51 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3425516143' entity='client.admin' 2023-11-16T23:58:52.202 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:51 smithi006 ceph-mon[97840]: from='mgr.14116 172.21.15.6:0/2266128508' entity='mgr.x' 2023-11-16T23:58:53.013 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:52 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3165042618' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-16T23:58:53.014 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:52 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setuser ceph since I am not root 2023-11-16T23:58:53.014 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:52 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setgroup ceph since I am not root 2023-11-16T23:58:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.182+0000 7fa10237d700 1 Processor -- start 2023-11-16T23:58:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.182+0000 7fa10237d700 1 -- start start 2023-11-16T23:58:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa10237d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa10237d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc07a7c0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60636/0 (socket says 172.21.15.6:60636) 2023-11-16T23:58:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa0fbfff700 1 -- 172.21.15.6:0/106713323 learned_addr learned my addr 172.21.15.6:0/106713323 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.183+0000 7fa0fbfff700 1 -- 172.21.15.6:0/106713323 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc07a900 con 0x7fa0fc07bd80 2023-11-16T23:58:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa0fbfff700 1 --2- 172.21.15.6:0/106713323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0098a0 tx=0x7fa0ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a838f77cc71ef93b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa0faffd700 1 -- 172.21.15.6:0/106713323 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec004160 con 0x7fa0fc07bd80 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa0faffd700 1 -- 172.21.15.6:0/106713323 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec0042c0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa0faffd700 1 -- 172.21.15.6:0/106713323 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec0119b0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa10237d700 1 -- 172.21.15.6:0/106713323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 msgr2=0x7fa0fc07a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.184+0000 7fa10237d700 1 --2- 172.21.15.6:0/106713323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0098a0 tx=0x7fa0ec009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.185+0000 7fa10237d700 1 -- 172.21.15.6:0/106713323 shutdown_connections 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.185+0000 7fa10237d700 1 --2- 172.21.15.6:0/106713323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc07a280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.185+0000 7fa10237d700 1 -- 172.21.15.6:0/106713323 >> 172.21.15.6:0/106713323 conn(0x7fa0fc100fe0 msgr2=0x7fa0fc1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.185+0000 7fa10237d700 1 -- 172.21.15.6:0/106713323 shutdown_connections 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.185+0000 7fa10237d700 1 -- 172.21.15.6:0/106713323 wait complete. 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.186+0000 7fa10237d700 1 Processor -- start 2023-11-16T23:58:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.186+0000 7fa10237d700 1 -- start start 2023-11-16T23:58:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.186+0000 7fa10237d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.186+0000 7fa10237d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19c040 con 0x7fa0fc07bd80 2023-11-16T23:58:53.387 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60640/0 (socket says 172.21.15.6:60640) 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0fbfff700 1 -- 172.21.15.6:0/3165042618 learned_addr learned my addr 172.21.15.6:0/3165042618 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0fbfff700 1 -- 172.21.15.6:0/3165042618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec009550 con 0x7fa0fc07bd80 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0fbfff700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec0039f0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.187+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec003b50 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.188+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc10be50 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.188+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec01a780 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.188+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc10bcc0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.189+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa0ec01a9f0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.189+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc10bfe0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.189+0000 7fa0f97fa700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 0x7fa0e403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.189+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa0ec04c9c0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.190+0000 7fa0fb7fe700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 0x7fa0e403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:53.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.190+0000 7fa0fb7fe700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 0x7fa0e403acd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006fd0 tx=0x7fa0f0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.194+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0ec01aca0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.468+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fa0fc19f4c0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.768+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa0ec02dc90 con 0x7fa0fc07bd80 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.770+0000 7fa0f97fa700 1 -- 172.21.15.6:0/3165042618 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fa0ec00b3e0 con 0x7fa0fc07bd80 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.772+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 msgr2=0x7fa0e403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.772+0000 7fa10237d700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 0x7fa0e403acd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006fd0 tx=0x7fa0f0006e40 comp rx=0 tx=0).stop 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.772+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 msgr2=0x7fa0fc19bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.772+0000 7fa10237d700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec011a30 comp rx=0 tx=0).stop 2023-11-16T23:58:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 shutdown_connections 2023-11-16T23:58:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fa0e4038820 0x7fa0e403acd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 --2- 172.21.15.6:0/3165042618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0fc07bd80 0x7fa0fc19bb00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 >> 172.21.15.6:0/3165042618 conn(0x7fa0fc100fe0 msgr2=0x7fa0fc101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 shutdown_connections 2023-11-16T23:58:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:52.773+0000 7fa10237d700 1 -- 172.21.15.6:0/3165042618 wait complete. 2023-11-16T23:58:53.448 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:53.081+0000 7fdc12fc2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T23:58:53.448 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:53.195+0000 7fdc12fc2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T23:58:53.449 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:53.314+0000 7fdc12fc2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T23:58:53.693 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:53.462+0000 7fdc12fc2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T23:58:53.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3165042618' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-16T23:58:53.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-mon[97840]: mgrmap e9: x(active, since 19s) 2023-11-16T23:58:53.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:53.693+0000 7fdc12fc2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T23:58:54.413 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:54 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:54.115+0000 7fdc12fc2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T23:58:54.451 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:54 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:54.235+0000 7fdc12fc2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T23:58:54.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:54 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:54.411+0000 7fdc12fc2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T23:58:54.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:54 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:54.541+0000 7fdc12fc2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T23:58:54.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:58:54.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-16T23:58:54.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T23:58:54.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T23:58:54.770 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.013+0000 7fb28b427700 1 Processor -- start 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.013+0000 7fb28b427700 1 -- start start 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.013+0000 7fb28b427700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.013+0000 7fb28b427700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28407a7c0 con 0x7fb28407bd80 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb2891c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb2891c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60666/0 (socket says 172.21.15.6:60666) 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb2891c3700 1 -- 172.21.15.6:0/3957184857 learned_addr learned my addr 172.21.15.6:0/3957184857 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb2891c3700 1 -- 172.21.15.6:0/3957184857 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28407a900 con 0x7fb28407bd80 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb2891c3700 1 --2- 172.21.15.6:0/3957184857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f743067745b2d4cf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.014+0000 7fb27bfff700 1 -- 172.21.15.6:0/3957184857 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280004160 con 0x7fb28407bd80 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb27bfff700 1 -- 172.21.15.6:0/3957184857 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2800042c0 con 0x7fb28407bd80 2023-11-16T23:58:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb27bfff700 1 -- 172.21.15.6:0/3957184857 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2800119b0 con 0x7fb28407bd80 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 -- 172.21.15.6:0/3957184857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 msgr2=0x7fb28407a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 --2- 172.21.15.6:0/3957184857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).stop 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 -- 172.21.15.6:0/3957184857 shutdown_connections 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 --2- 172.21.15.6:0/3957184857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28407a280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 -- 172.21.15.6:0/3957184857 >> 172.21.15.6:0/3957184857 conn(0x7fb284100fe0 msgr2=0x7fb2841033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 -- 172.21.15.6:0/3957184857 shutdown_connections 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.015+0000 7fb28b427700 1 -- 172.21.15.6:0/3957184857 wait complete. 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.016+0000 7fb28b427700 1 Processor -- start 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.016+0000 7fb28b427700 1 -- start start 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb28b427700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb28b427700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419c040 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb2891c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb2891c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60674/0 (socket says 172.21.15.6:60674) 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb2891c3700 1 -- 172.21.15.6:0/2845758478 learned_addr learned my addr 172.21.15.6:0/2845758478 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.017+0000 7fb2891c3700 1 -- 172.21.15.6:0/2845758478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009550 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb2891c3700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2800039f0 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb280003b50 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb28001a780 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28410be50 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.018+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28410bcc0 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.019+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fb280024440 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.019+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28410bfe0 con 0x7fb28407bd80 2023-11-16T23:58:54.773 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.020+0000 7fb27a7fc700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 0x7fb27003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.020+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb28004c7c0 con 0x7fb28407bd80 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.020+0000 7fb2889c2700 1 -- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 msgr2=0x7fb27003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.020+0000 7fb2889c2700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 0x7fb27003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.025+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb2800189a0 con 0x7fb28407bd80 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.221+0000 7fb2889c2700 1 -- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 msgr2=0x7fb27003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.221+0000 7fb2889c2700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 0x7fb27003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.291+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb28419f4c0 con 0x7fb28407bd80 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.292+0000 7fb27a7fc700 1 -- 172.21.15.6:0/2845758478 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb280018350 con 0x7fb28407bd80 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 msgr2=0x7fb27003ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 0x7fb27003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 msgr2=0x7fb28419bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280011a30 comp rx=0 tx=0).stop 2023-11-16T23:58:54.774 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 shutdown_connections 2023-11-16T23:58:54.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7fb270038870 0x7fb27003ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:54.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 --2- 172.21.15.6:0/2845758478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb28407bd80 0x7fb28419bb00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:58:54.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 >> 172.21.15.6:0/2845758478 conn(0x7fb284100fe0 msgr2=0x7fb284101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:58:54.775 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 shutdown_connections 2023-11-16T23:58:54.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:54.294+0000 7fb28b427700 1 -- 172.21.15.6:0/2845758478 wait complete. 2023-11-16T23:58:54.814 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2023-11-16T23:58:55.810 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2023-11-16T23:58:55.811 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:58:54 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2845758478' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T23:58:56.001 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:55 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:55.116+0000 7fdc12fc2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T23:58:56.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:55 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:55.903+0000 7fdc12fc2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T23:58:56.223 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:56 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:56.117+0000 7fdc12fc2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T23:58:57.464 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:57.106+0000 7fdc12fc2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:59:10.153 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:57.231+0000 7fdc12fc2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T23:59:10.154 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:57.463+0000 7fdc12fc2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T23:59:10.154 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:57.841+0000 7fdc12fc2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T23:59:10.154 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:58.125+0000 7fdc12fc2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T23:59:10.154 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:58.326+0000 7fdc12fc2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T23:59:10.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:58.686+0000 7fdc12fc2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T23:59:10.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:59 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:59.662+0000 7fdc12fc2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T23:59:10.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:58:59 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:58:59.842+0000 7fdc12fc2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T23:59:10.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:59:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:59:00.101+0000 7fdc12fc2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T23:59:10.155 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:59:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:59:00.256+0000 7fdc12fc2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T23:59:10.259 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:59:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:59:00.544+0000 7fdc12fc2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T23:59:10.259 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:59:01.651+0000 7fdc12fc2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T23:59:10.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: Active manager daemon x restarted 2023-11-16T23:59:10.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: Activating manager daemon x 2023-11-16T23:59:10.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: osdmap e3: 0 total, 0 up, 0 in 2023-11-16T23:59:10.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: mgrmap e10: x(active, starting, since 0.00396032s) 2023-11-16T23:59:10.383 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:59:10.384 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T23:59:10.388 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T23:59:10.389 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T23:59:10.389 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T23:59:10.390 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: Manager daemon x is now available 2023-11-16T23:59:10.391 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:10.392 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:10.392 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T23:59:10.393 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T23:59:10.393 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:02 smithi006 ceph-mon[97840]: mgrmap e11: x(active, since 1.00885s) 2023-11-16T23:59:10.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2023-11-16T23:59:10.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-16T23:59:10.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T23:59:10.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2023-11-16T23:59:10.406 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.381+0000 7f215db32700 1 Processor -- start 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.381+0000 7f215db32700 1 -- start start 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.381+0000 7f215db32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.381+0000 7f215db32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21581062a0 con 0x7f2158105950 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.382+0000 7f21577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.382+0000 7f21577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60680/0 (socket says 172.21.15.6:60680) 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.382+0000 7f21577fe700 1 -- 172.21.15.6:0/3105219172 learned_addr learned my addr 172.21.15.6:0/3105219172 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.407 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.382+0000 7f21577fe700 1 -- 172.21.15.6:0/3105219172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21581063e0 con 0x7f2158105950 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.382+0000 7f21577fe700 1 --2- 172.21.15.6:0/3105219172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21480098a0 tx=0x7f2148009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc2ee5b0654c87 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f21567fc700 1 -- 172.21.15.6:0/3105219172 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2148004160 con 0x7f2158105950 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f21567fc700 1 -- 172.21.15.6:0/3105219172 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21480042c0 con 0x7f2158105950 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f21567fc700 1 -- 172.21.15.6:0/3105219172 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21480119b0 con 0x7f2158105950 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 -- 172.21.15.6:0/3105219172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 msgr2=0x7f2158105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 --2- 172.21.15.6:0/3105219172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21480098a0 tx=0x7f2148009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 -- 172.21.15.6:0/3105219172 shutdown_connections 2023-11-16T23:59:10.408 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 --2- 172.21.15.6:0/3105219172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f2158105d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 -- 172.21.15.6:0/3105219172 >> 172.21.15.6:0/3105219172 conn(0x7f2158100fe0 msgr2=0x7f21581033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.383+0000 7f215db32700 1 -- 172.21.15.6:0/3105219172 shutdown_connections 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.384+0000 7f215db32700 1 -- 172.21.15.6:0/3105219172 wait complete. 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.384+0000 7f215db32700 1 Processor -- start 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f215db32700 1 -- start start 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f215db32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f215db32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158199de0 con 0x7f2158105950 2023-11-16T23:59:10.409 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f21577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f21577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60682/0 (socket says 172.21.15.6:60682) 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.385+0000 7f21577fe700 1 -- 172.21.15.6:0/1360636536 learned_addr learned my addr 172.21.15.6:0/1360636536 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f21577fe700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2148009550 con 0x7f2158105950 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f21577fe700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2148011980 tx=0x7f2148011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2148003a60 con 0x7f2158105950 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2148003bc0 con 0x7f2158105950 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214801a670 con 0x7f2158105950 2023-11-16T23:59:10.410 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21581079e0 con 0x7f2158105950 2023-11-16T23:59:10.411 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.386+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2158107850 con 0x7f2158105950 2023-11-16T23:59:10.411 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.387+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f2148024a00 con 0x7f2158105950 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.388+0000 7f2154ff9700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.388+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2158109e20 con 0x7f2140038870 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.388+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f214804c930 con 0x7f2158105950 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.388+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.388+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.589+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.589+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T23:59:10.412 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.990+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:55.990+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:56.791+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:56.791+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:58.392+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:58:58.393+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:01.596+0000 7f2156ffd700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/610057204 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:01.596+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:01.655+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f214802d910 con 0x7f2158105950 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:01.655+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 msgr2=0x7f214003ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:01.655+0000 7f2154ff9700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.659+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f214804e200 con 0x7f2158105950 2023-11-16T23:59:10.413 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.660+0000 7f2154ff9700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 0x7f214003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.661+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2158109e20 con 0x7f214003bfe0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.661+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 0x7f214003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.663+0000 7f2156ffd700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 0x7f214003e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f214c003a10 tx=0x7f214c009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.664+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2158109e20 con 0x7f214003bfe0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.671+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f215819abe0 con 0x7f214003bfe0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.671+0000 7f2154ff9700 1 -- 172.21.15.6:0/1360636536 <== mgr.14148 v2:172.21.15.6:6800/879660796 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f215819abe0 con 0x7f214003bfe0 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.672+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 msgr2=0x7f214003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.672+0000 7f215db32700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 0x7f214003e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f214c003a10 tx=0x7f214c009270 comp rx=0 tx=0).stop 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.672+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 msgr2=0x7f21581998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.672+0000 7f215db32700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2148011980 tx=0x7f2148011a60 comp rx=0 tx=0).stop 2023-11-16T23:59:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 shutdown_connections 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f214003bfe0 0x7f214003e3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2158105950 0x7f21581998a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 --2- 172.21.15.6:0/1360636536 >> [v2:172.21.15.6:6800/610057204,v1:172.21.15.6:6801/610057204] conn(0x7f2140038870 0x7f214003ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 >> 172.21.15.6:0/1360636536 conn(0x7f2158100fe0 msgr2=0x7f2158101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 shutdown_connections 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:02.673+0000 7f215db32700 1 -- 172.21.15.6:0/1360636536 wait complete. 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2023-11-16T23:59:10.415 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2023-11-16T23:59:10.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:03 smithi006 ceph-mon[97840]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-16T23:59:10.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:03 smithi006 ceph-mon[97840]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-16T23:59:10.423 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.861+0000 7f635e166700 1 Processor -- start 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.861+0000 7f635e166700 1 -- start start 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.862+0000 7f635e166700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.862+0000 7f635e166700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635807a840 con 0x7f635807be00 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.862+0000 7f63577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.424 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50220/0 (socket says 172.21.15.6:50220) 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63577fe700 1 -- 172.21.15.6:0/1311232498 learned_addr learned my addr 172.21.15.6:0/1311232498 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63577fe700 1 -- 172.21.15.6:0/1311232498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635807a980 con 0x7f635807be00 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63577fe700 1 --2- 172.21.15.6:0/1311232498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63480098a0 tx=0x7f6348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27d57a09c1529c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63567fc700 1 -- 172.21.15.6:0/1311232498 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6348004160 con 0x7f635807be00 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63567fc700 1 -- 172.21.15.6:0/1311232498 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63480042c0 con 0x7f635807be00 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.863+0000 7f63567fc700 1 -- 172.21.15.6:0/1311232498 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63480119b0 con 0x7f635807be00 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 -- 172.21.15.6:0/1311232498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 msgr2=0x7f635807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.425 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 --2- 172.21.15.6:0/1311232498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63480098a0 tx=0x7f6348009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 -- 172.21.15.6:0/1311232498 shutdown_connections 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 --2- 172.21.15.6:0/1311232498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635807a300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 -- 172.21.15.6:0/1311232498 >> 172.21.15.6:0/1311232498 conn(0x7f6358100fe0 msgr2=0x7f63581033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 -- 172.21.15.6:0/1311232498 shutdown_connections 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.864+0000 7f635e166700 1 -- 172.21.15.6:0/1311232498 wait complete. 2023-11-16T23:59:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f635e166700 1 Processor -- start 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f635e166700 1 -- start start 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f635e166700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f635e166700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635819c050 con 0x7f635807be00 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f63577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f63577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50222/0 (socket says 172.21.15.6:50222) 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.866+0000 7f63577fe700 1 -- 172.21.15.6:0/3463198826 learned_addr learned my addr 172.21.15.6:0/3463198826 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f63577fe700 1 -- 172.21.15.6:0/3463198826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009550 con 0x7f635807be00 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f63577fe700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6348003e60 tx=0x7f6348011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6348003970 con 0x7f635807be00 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6348003ad0 con 0x7f635807be00 2023-11-16T23:59:10.427 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635810be30 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.867+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63480195e0 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.868+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635810bca0 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.869+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635810bfc0 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.869+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f6348019850 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.870+0000 7f6354ff9700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 0x7f634003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.870+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f634804c9e0 con 0x7f635807be00 2023-11-16T23:59:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.870+0000 7f6356ffd700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 0x7f634003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.871+0000 7f6356ffd700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 0x7f634003acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f634c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:03.876+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6348003c40 con 0x7f635807be00 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.097+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f635819c600 con 0x7f6340038820 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.281+0000 7f6354ff9700 1 -- 172.21.15.6:0/3463198826 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f635819c600 con 0x7f6340038820 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 msgr2=0x7f634003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 0x7f634003acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f634c006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 msgr2=0x7f635819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6348003e60 tx=0x7f6348011aa0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 shutdown_connections 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6340038820 0x7f634003acd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 --2- 172.21.15.6:0/3463198826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635807be00 0x7f635819bb10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 >> 172.21.15.6:0/3463198826 conn(0x7f6358100fe0 msgr2=0x7f6358101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 shutdown_connections 2023-11-16T23:59:10.430 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:04.283+0000 7f635e166700 1 -- 172.21.15.6:0/3463198826 wait complete. 2023-11-16T23:59:10.430 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2023-11-16T23:59:10.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: [16/Nov/2023:23:59:03] ENGINE Bus STARTING 2023-11-16T23:59:10.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: [16/Nov/2023:23:59:03] ENGINE Serving on http://172.21.15.6:8765 2023-11-16T23:59:10.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: [16/Nov/2023:23:59:03] ENGINE Serving on https://172.21.15.6:7150 2023-11-16T23:59:10.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: [16/Nov/2023:23:59:03] ENGINE Bus STARTED 2023-11-16T23:59:10.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: mgrmap e12: x(active, since 2s) 2023-11-16T23:59:10.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:10.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:10.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:10.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:05 smithi006 ceph-mon[97840]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:10.440 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$W66MmUKLye6fsawpZrXjpeT4CDPql8gI6EiQn4cFqWCN/tjAvK9GO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1700179145, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-16T23:59:10.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6187f6a700 1 Processor -- start 2023-11-16T23:59:10.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6187f6a700 1 -- start start 2023-11-16T23:59:10.874 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6187f6a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6187f6a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618007a7c0 con 0x7f618007bd80 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6185d06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.514+0000 7f6185d06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50226/0 (socket says 172.21.15.6:50226) 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6185d06700 1 -- 172.21.15.6:0/4049675515 learned_addr learned my addr 172.21.15.6:0/4049675515 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6185d06700 1 -- 172.21.15.6:0/4049675515 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618007a900 con 0x7f618007bd80 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6185d06700 1 --2- 172.21.15.6:0/4049675515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098a0 tx=0x7f617c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78353b18fca2a3c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6184d04700 1 -- 172.21.15.6:0/4049675515 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f617c004160 con 0x7f618007bd80 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6184d04700 1 -- 172.21.15.6:0/4049675515 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617c0042c0 con 0x7f618007bd80 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.515+0000 7f6184d04700 1 -- 172.21.15.6:0/4049675515 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f617c0119b0 con 0x7f618007bd80 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 -- 172.21.15.6:0/4049675515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 msgr2=0x7f618007a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 --2- 172.21.15.6:0/4049675515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098a0 tx=0x7f617c009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.875 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 -- 172.21.15.6:0/4049675515 shutdown_connections 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 --2- 172.21.15.6:0/4049675515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618007a280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 -- 172.21.15.6:0/4049675515 >> 172.21.15.6:0/4049675515 conn(0x7f6180100fe0 msgr2=0x7f61801033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 -- 172.21.15.6:0/4049675515 shutdown_connections 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.516+0000 7f6187f6a700 1 -- 172.21.15.6:0/4049675515 wait complete. 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.517+0000 7f6187f6a700 1 Processor -- start 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6187f6a700 1 -- start start 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6187f6a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6187f6a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618019c050 con 0x7f618007bd80 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6185d06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6185d06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50232/0 (socket says 172.21.15.6:50232) 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.518+0000 7f6185d06700 1 -- 172.21.15.6:0/2212114643 learned_addr learned my addr 172.21.15.6:0/2212114643 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6185d06700 1 -- 172.21.15.6:0/2212114643 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c009550 con 0x7f618007bd80 2023-11-16T23:59:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6185d06700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f617c000c00 tx=0x7f617c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f617c0039f0 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617c003b50 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6187f6a700 1 -- 172.21.15.6:0/2212114643 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618010be50 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f617c01a780 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.519+0000 7f6187f6a700 1 -- 172.21.15.6:0/2212114643 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618010bcc0 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.520+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f617c01a9f0 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.521+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618010bfe0 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.521+0000 7f6176ffd700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 0x7f616c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.521+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f617c04c9b0 con 0x7f618007bd80 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.521+0000 7f6185505700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 0x7f616c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.522+0000 7f6185505700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 0x7f616c03ad70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6170006fd0 tx=0x7f6170006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.877 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.526+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f617c0237b0 con 0x7f618007bd80 2023-11-16T23:59:10.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:05.753+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f618004e980 con 0x7f616c0388c0 2023-11-16T23:59:10.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.003+0000 7f6176ffd700 1 -- 172.21.15.6:0/2212114643 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f618004e980 con 0x7f616c0388c0 2023-11-16T23:59:10.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.005+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 msgr2=0x7f616c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.878 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.005+0000 7f6174ff9700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 0x7f616c03ad70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6170006fd0 tx=0x7f6170006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:10.879 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.005+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 msgr2=0x7f618019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.912 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.005+0000 7f6174ff9700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f617c000c00 tx=0x7f617c011a30 comp rx=0 tx=0).stop 2023-11-16T23:59:10.956 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 shutdown_connections 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f616c0388c0 0x7f616c03ad70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 --2- 172.21.15.6:0/2212114643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618007bd80 0x7f618019bb10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 >> 172.21.15.6:0/2212114643 conn(0x7f6180100fe0 msgr2=0x7f6180101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 shutdown_connections 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:06.006+0000 7f6174ff9700 1 -- 172.21.15.6:0/2212114643 wait complete. 2023-11-16T23:59:10.971 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2023-11-16T23:59:10.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:06 smithi006 ceph-mon[97840]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:10.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:10.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:10.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:07 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/35814751' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-16T23:59:10.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2023-11-16T23:59:10.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.438+0000 7fa695c02700 1 Processor -- start 2023-11-16T23:59:10.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa695c02700 1 -- start start 2023-11-16T23:59:10.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa695c02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.990 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa695c02700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6901062a0 con 0x7fa690105950 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa68f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa68f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50238/0 (socket says 172.21.15.6:50238) 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.439+0000 7fa68f7fe700 1 -- 172.21.15.6:0/235654381 learned_addr learned my addr 172.21.15.6:0/235654381 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.440+0000 7fa68f7fe700 1 -- 172.21.15.6:0/235654381 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6901063e0 con 0x7fa690105950 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.440+0000 7fa68f7fe700 1 --2- 172.21.15.6:0/235654381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6800098a0 tx=0x7fa680009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2eb8a039fc3ee0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.440+0000 7fa68e7fc700 1 -- 172.21.15.6:0/235654381 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa680004160 con 0x7fa690105950 2023-11-16T23:59:10.991 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.440+0000 7fa68e7fc700 1 -- 172.21.15.6:0/235654381 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6800042c0 con 0x7fa690105950 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.440+0000 7fa68e7fc700 1 -- 172.21.15.6:0/235654381 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6800119b0 con 0x7fa690105950 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 -- 172.21.15.6:0/235654381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 msgr2=0x7fa690105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 --2- 172.21.15.6:0/235654381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6800098a0 tx=0x7fa680009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 -- 172.21.15.6:0/235654381 shutdown_connections 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 --2- 172.21.15.6:0/235654381 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa690105d60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 -- 172.21.15.6:0/235654381 >> 172.21.15.6:0/235654381 conn(0x7fa690100fe0 msgr2=0x7fa6901033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.992 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 -- 172.21.15.6:0/235654381 shutdown_connections 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.441+0000 7fa695c02700 1 -- 172.21.15.6:0/235654381 wait complete. 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.443+0000 7fa695c02700 1 Processor -- start 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.443+0000 7fa695c02700 1 -- start start 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.443+0000 7fa695c02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.443+0000 7fa695c02700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690199df0 con 0x7fa690105950 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.444+0000 7fa68f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.444+0000 7fa68f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50254/0 (socket says 172.21.15.6:50254) 2023-11-16T23:59:10.993 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.444+0000 7fa68f7fe700 1 -- 172.21.15.6:0/35814751 learned_addr learned my addr 172.21.15.6:0/35814751 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.444+0000 7fa68f7fe700 1 -- 172.21.15.6:0/35814751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680009550 con 0x7fa690105950 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.444+0000 7fa68f7fe700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa680011980 tx=0x7fa680011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.445+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa680003a60 con 0x7fa690105950 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.445+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa680003bc0 con 0x7fa690105950 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.445+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa68001a670 con 0x7fa690105950 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.445+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6901079e0 con 0x7fa690105950 2023-11-16T23:59:10.994 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.445+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa690107850 con 0x7fa690105950 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.446+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa680024a00 con 0x7fa690105950 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.446+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa690109e20 con 0x7fa690105950 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.446+0000 7fa68cff9700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 0x7fa67803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.446+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa68004ca10 con 0x7fa690105950 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.446+0000 7fa68effd700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 0x7fa67803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.447+0000 7fa68effd700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 0x7fa67803ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa684006fd0 tx=0x7fa684006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.451+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa680024440 con 0x7fa690105950 2023-11-16T23:59:10.995 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.660+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa69019a330 con 0x7fa690105950 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.661+0000 7fa68cff9700 1 -- 172.21.15.6:0/35814751 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa680018350 con 0x7fa690105950 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 msgr2=0x7fa67803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 0x7fa67803ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa684006fd0 tx=0x7fa684006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 msgr2=0x7fa6901998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa680011980 tx=0x7fa680011a60 comp rx=0 tx=0).stop 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 shutdown_connections 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa678038870 0x7fa67803ad20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 --2- 172.21.15.6:0/35814751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa690105950 0x7fa6901998b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 >> 172.21.15.6:0/35814751 conn(0x7fa690100fe0 msgr2=0x7fa690101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 shutdown_connections 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:07.663+0000 7fa695c02700 1 -- 172.21.15.6:0/35814751 wait complete. 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2023-11-16T23:59:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout: Password: xit8epezz3 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config directory 2023-11-16T23:59:10.998 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling autotune for osd_memory_target 2023-11-16T23:59:10.999 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:08 smithi006 ceph-mon[97840]: mgrmap e13: x(active, since 6s) 2023-11-16T23:59:11.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.396+0000 7f4174ccd700 1 Processor -- start 2023-11-16T23:59:11.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.396+0000 7f4174ccd700 1 -- start start 2023-11-16T23:59:11.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.396+0000 7f4174ccd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.396+0000 7f4174ccd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417007a7c0 con 0x7f417007bd80 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.397+0000 7f416e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.397+0000 7f416e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54910/0 (socket says 172.21.15.6:54910) 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.397+0000 7f416e59c700 1 -- 172.21.15.6:0/480869939 learned_addr learned my addr 172.21.15.6:0/480869939 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.397+0000 7f416e59c700 1 -- 172.21.15.6:0/480869939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417007a900 con 0x7f417007bd80 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.397+0000 7f416e59c700 1 --2- 172.21.15.6:0/480869939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b84b47ebcba3c20 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f416d59a700 1 -- 172.21.15.6:0/480869939 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4160004160 con 0x7f417007bd80 2023-11-16T23:59:11.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f416d59a700 1 -- 172.21.15.6:0/480869939 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41600042c0 con 0x7f417007bd80 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f416d59a700 1 -- 172.21.15.6:0/480869939 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41600119b0 con 0x7f417007bd80 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 -- 172.21.15.6:0/480869939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 msgr2=0x7f417007a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 --2- 172.21.15.6:0/480869939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 -- 172.21.15.6:0/480869939 shutdown_connections 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 --2- 172.21.15.6:0/480869939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417007a280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 -- 172.21.15.6:0/480869939 >> 172.21.15.6:0/480869939 conn(0x7f4170100fe0 msgr2=0x7f41701033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:11.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.398+0000 7f4174ccd700 1 -- 172.21.15.6:0/480869939 shutdown_connections 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.399+0000 7f4174ccd700 1 -- 172.21.15.6:0/480869939 wait complete. 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.399+0000 7f4174ccd700 1 Processor -- start 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f4174ccd700 1 -- start start 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f4174ccd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f4174ccd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417019c040 con 0x7f417007bd80 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f416e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f416e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54920/0 (socket says 172.21.15.6:54920) 2023-11-16T23:59:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f416e59c700 1 -- 172.21.15.6:0/4058859760 learned_addr learned my addr 172.21.15.6:0/4058859760 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.400+0000 7f416e59c700 1 -- 172.21.15.6:0/4058859760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009550 con 0x7f417007bd80 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f416e59c700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41600039f0 con 0x7f417007bd80 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4160003b50 con 0x7f417007bd80 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416001a780 con 0x7f417007bd80 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417010be50 con 0x7f417007bd80 2023-11-16T23:59:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.401+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417010bcc0 con 0x7f417007bd80 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.402+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f4160024440 con 0x7f417007bd80 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.402+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417010bfe0 con 0x7f417007bd80 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.403+0000 7f415f7fe700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 0x7f415803adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.403+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f416004c9c0 con 0x7f417007bd80 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.403+0000 7f416dd9b700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 0x7f415803adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.403+0000 7f416dd9b700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 0x7f415803adc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.408+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4160018d90 con 0x7f417007bd80 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.615+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f417019c580 con 0x7f417007bd80 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.615+0000 7f415f7fe700 1 -- 172.21.15.6:0/4058859760 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4160018350 con 0x7f417007bd80 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 msgr2=0x7f415803adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 0x7f415803adc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 msgr2=0x7f417019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011a30 comp rx=0 tx=0).stop 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 shutdown_connections 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4158038910 0x7f415803adc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 --2- 172.21.15.6:0/4058859760 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f417007bd80 0x7f417019bb00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.617+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 >> 172.21.15.6:0/4058859760 conn(0x7f4170100fe0 msgr2=0x7f4170101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.618+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 shutdown_connections 2023-11-16T23:59:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:09.618+0000 7f4174ccd700 1 -- 172.21.15.6:0/4058859760 wait complete. 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.755+0000 7f4ddb5a6700 1 Processor -- start 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.755+0000 7f4ddb5a6700 1 -- start start 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.755+0000 7f4ddb5a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.755+0000 7f4ddb5a6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd407a7e0 con 0x7f4dd407bda0 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.755+0000 7f4dd9342700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dd9342700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54930/0 (socket says 172.21.15.6:54930) 2023-11-16T23:59:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dd9342700 1 -- 172.21.15.6:0/2533751644 learned_addr learned my addr 172.21.15.6:0/2533751644 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dd9342700 1 -- 172.21.15.6:0/2533751644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd407a920 con 0x7f4dd407bda0 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dd9342700 1 --2- 172.21.15.6:0/2533751644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098a0 tx=0x7f4dd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f062d3de0effa0e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dcbfff700 1 -- 172.21.15.6:0/2533751644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0004160 con 0x7f4dd407bda0 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dcbfff700 1 -- 172.21.15.6:0/2533751644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd00042c0 con 0x7f4dd407bda0 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.756+0000 7f4dcbfff700 1 -- 172.21.15.6:0/2533751644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd00119b0 con 0x7f4dd407bda0 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/2533751644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 msgr2=0x7f4dd407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.645 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/2533751644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00098a0 tx=0x7f4dd0009bb0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/2533751644 shutdown_connections 2023-11-16T23:59:11.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/2533751644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd407a2a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/2533751644 >> 172.21.15.6:0/2533751644 conn(0x7f4dd4100fe0 msgr2=0x7f4dd41033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:11.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/2533751644 shutdown_connections 2023-11-16T23:59:11.648 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.757+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/2533751644 wait complete. 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.758+0000 7f4ddb5a6700 1 Processor -- start 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.758+0000 7f4ddb5a6700 1 -- start start 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4ddb5a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4ddb5a6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd419c010 con 0x7f4dd407bda0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4dd9342700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4dd9342700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54940/0 (socket says 172.21.15.6:54940) 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4dd9342700 1 -- 172.21.15.6:0/4279366474 learned_addr learned my addr 172.21.15.6:0/4279366474 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4dd9342700 1 -- 172.21.15.6:0/4279366474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0009550 con 0x7f4dd407bda0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.759+0000 7f4dd9342700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0003e60 tx=0x7f4dd0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.760+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd0003970 con 0x7f4dd407bda0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.760+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd0003ad0 con 0x7f4dd407bda0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.760+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd00195e0 con 0x7f4dd407bda0 2023-11-16T23:59:11.649 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.760+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd410be90 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.760+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd410bd00 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.761+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd410c020 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.761+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f4dd0019850 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.761+0000 7f4dca7fc700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 0x7f4dc003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.761+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4dd004cac0 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.762+0000 7f4dd8b41700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 0x7f4dc003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.762+0000 7f4dd8b41700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 0x7f4dc003ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4006fd0 tx=0x7f4dc4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:10.766+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4dd0003c40 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.083+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f4dd4062120 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.086+0000 7f4dca7fc700 1 -- 172.21.15.6:0/4279366474 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4dd0023b40 con 0x7f4dd407bda0 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 msgr2=0x7f4dc003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.650 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 0x7f4dc003ad70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4006fd0 tx=0x7f4dc4006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:11.651 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 msgr2=0x7f4dd419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:11.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0003e60 tx=0x7f4dd0011aa0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 shutdown_connections 2023-11-16T23:59:11.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4dc00388c0 0x7f4dc003ad70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 --2- 172.21.15.6:0/4279366474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dd407bda0 0x7f4dd419bad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 >> 172.21.15.6:0/4279366474 conn(0x7f4dd4100fe0 msgr2=0x7f4dd4101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 shutdown_connections 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2023-11-16T23:59:11.088+0000 7f4ddb5a6700 1 -- 172.21.15.6:0/4279366474 wait complete. 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2023-11-16T23:59:11.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-16T23:59:11.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:11.656 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2023-11-16T23:59:11.710 INFO:tasks.cephadm:Fetching config... 2023-11-16T23:59:11.711 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:59:11.714 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-16T23:59:11.760 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-16T23:59:11.760 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:59:11.760 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-16T23:59:11.847 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-16T23:59:11.847 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:59:11.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-11-16T23:59:11.959 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-16T23:59:11.959 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-16T23:59:11.959 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-16T23:59:12.045 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-16T23:59:12.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T23:59:12.192 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:12.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:12 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4279366474' entity='client.admin' 2023-11-16T23:59:12.222 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T23:59:12.267 INFO:teuthology.orchestra.run.smithi087.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:12.280 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T23:59:12.326 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsTs3CsuDrTFBnfLjeOAtCUR5SVuijPMXXLHUdR2xnfKgpjNp2YxNuYgx8tcpkCoeeZSuIsd5pR1StUrDJC53kp5PcEh3Gmg5mLS2JdzW7BKREEIVCMrgyq3H+uJU51cM88T6XnFCWwZpsAn6FreOpZzC5/5cf7HLBEwnpIl0pj4b5o0Xqn6EPfgsIVNGIFBlEJQ5OzaPeQwDHS6aVIQAaLR50UQJRnMtjetfF+qihc1szL/6Tk6WJVMSmacOwx/YvOopFb22FjTSuqRqJLouFQZxq6Uk71LBMRyc5PHDbQKISCkFifPssQZ3oldtJTo4K03HLTql9qxpgrdvqctk6VRfIwQeajD1FCh18NRsf4f5MhncxABdo2k43/PUQ1ljvWR7TwxTzcEpJysH/PCAmfrYq5PFWLZrOVc3zXUveDWRavQdI9YfYZ2NUofczl9v4Cnvkvp773mGzErAOXvx/TvVkDy5bJN3vm42bfxpby2JwIW6srzoJxagEK01nZ88= ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:12.339 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-16T23:59:13.535 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:14.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:14.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:15.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 -- 172.21.15.6:0/2056893377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 msgr2=0x7f58d4100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:15.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 --2- 172.21.15.6:0/2056893377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4100110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009910 tx=0x7f58c8009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:15.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 -- 172.21.15.6:0/2056893377 shutdown_connections 2023-11-16T23:59:15.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 --2- 172.21.15.6:0/2056893377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4100110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:15.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 -- 172.21.15.6:0/2056893377 >> 172.21.15.6:0/2056893377 conn(0x7f58d40fb2b0 msgr2=0x7f58d40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:15.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 -- 172.21.15.6:0/2056893377 shutdown_connections 2023-11-16T23:59:15.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.417+0000 7f58dab4e700 1 -- 172.21.15.6:0/2056893377 wait complete. 2023-11-16T23:59:15.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.419+0000 7f58dab4e700 1 Processor -- start 2023-11-16T23:59:15.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.419+0000 7f58dab4e700 1 -- start start 2023-11-16T23:59:15.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.419+0000 7f58dab4e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:15.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58dab4e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4195340 con 0x7f58d40ffd00 2023-11-16T23:59:15.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58d88ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:15.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58d88ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54952/0 (socket says 172.21.15.6:54952) 2023-11-16T23:59:15.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58d88ea700 1 -- 172.21.15.6:0/1201898706 learned_addr learned my addr 172.21.15.6:0/1201898706 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58d88ea700 1 -- 172.21.15.6:0/1201898706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80095f0 con 0x7f58d40ffd00 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.420+0000 7f58d88ea700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58c8004f40 tx=0x7f58c80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.421+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c801d070 con 0x7f58d40ffd00 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.421+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c800bc10 con 0x7f58d40ffd00 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.421+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d4105bd0 con 0x7f58d40ffd00 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.421+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c8003a70 con 0x7f58d40ffd00 2023-11-16T23:59:15.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.421+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d4105a40 con 0x7f58d40ffd00 2023-11-16T23:59:15.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.422+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d4105d60 con 0x7f58d40ffd00 2023-11-16T23:59:15.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.422+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f58c8003ce0 con 0x7f58d40ffd00 2023-11-16T23:59:15.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.422+0000 7f58d1ffb700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 0x7f58c403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:15.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.423+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58c804c550 con 0x7f58d40ffd00 2023-11-16T23:59:15.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.423+0000 7f58d3fff700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 0x7f58c403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:15.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.423+0000 7f58d3fff700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 0x7f58c403ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:15.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.428+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f58c802abf0 con 0x7f58d40ffd00 2023-11-16T23:59:15.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.635+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f58d418e900 con 0x7f58d40ffd00 2023-11-16T23:59:15.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.639+0000 7f58d1ffb700 1 -- 172.21.15.6:0/1201898706 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f58c8026070 con 0x7f58d40ffd00 2023-11-16T23:59:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 msgr2=0x7f58c403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 0x7f58c403ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 msgr2=0x7f58d4194e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58c8004f40 tx=0x7f58c80057c0 comp rx=0 tx=0).stop 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 shutdown_connections 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f58c40388c0 0x7f58c403ad70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 --2- 172.21.15.6:0/1201898706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58d40ffd00 0x7f58d4194e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 >> 172.21.15.6:0/1201898706 conn(0x7f58d40fb2b0 msgr2=0x7f58d40fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 shutdown_connections 2023-11-16T23:59:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:15.641+0000 7f58dab4e700 1 -- 172.21.15.6:0/1201898706 wait complete. 2023-11-16T23:59:16.223 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-16T23:59:16.223 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-16T23:59:16.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1201898706' entity='client.admin' 2023-11-16T23:59:16.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:16.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:17.080 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:18.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.866+0000 7fd570ebe700 1 -- 172.21.15.6:0/4218612176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 msgr2=0x7fd56c100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.866+0000 7fd570ebe700 1 --2- 172.21.15.6:0/4218612176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c100110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009910 tx=0x7fd55c009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.867+0000 7fd570ebe700 1 -- 172.21.15.6:0/4218612176 shutdown_connections 2023-11-16T23:59:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.867+0000 7fd570ebe700 1 --2- 172.21.15.6:0/4218612176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c100110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.867+0000 7fd570ebe700 1 -- 172.21.15.6:0/4218612176 >> 172.21.15.6:0/4218612176 conn(0x7fd56c0fb2b0 msgr2=0x7fd56c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.867+0000 7fd570ebe700 1 -- 172.21.15.6:0/4218612176 shutdown_connections 2023-11-16T23:59:18.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.867+0000 7fd570ebe700 1 -- 172.21.15.6:0/4218612176 wait complete. 2023-11-16T23:59:18.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.868+0000 7fd570ebe700 1 Processor -- start 2023-11-16T23:59:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.868+0000 7fd570ebe700 1 -- start start 2023-11-16T23:59:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.868+0000 7fd570ebe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.868+0000 7fd570ebe700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c197530 con 0x7fd56c0ffd00 2023-11-16T23:59:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd56a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd56a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54058/0 (socket says 172.21.15.6:54058) 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd56a59c700 1 -- 172.21.15.6:0/2947818731 learned_addr learned my addr 172.21.15.6:0/2947818731 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd56a59c700 1 -- 172.21.15.6:0/2947818731 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0095f0 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd56a59c700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c004f40 tx=0x7fd55c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd55c01d070 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55c00bc10 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c105bd0 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.869+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd55c003a70 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.870+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c105a40 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.871+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fd55c003ce0 con 0x7fd56c0ffd00 2023-11-16T23:59:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.871+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c105d60 con 0x7fd56c0ffd00 2023-11-16T23:59:18.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.871+0000 7fd55b7fe700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 0x7fd55403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:18.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.871+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd55c04c550 con 0x7fd56c0ffd00 2023-11-16T23:59:18.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.871+0000 7fd569d9b700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 0x7fd55403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:18.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.872+0000 7fd569d9b700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 0x7fd55403ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd560006fd0 tx=0x7fd560006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:18.876+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd55c02abf0 con 0x7fd56c0ffd00 2023-11-16T23:59:19.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.086+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd56c060f30 con 0x7fd5540388c0 2023-11-16T23:59:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.089+0000 7fd55b7fe700 1 -- 172.21.15.6:0/2947818731 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd56c060f30 con 0x7fd5540388c0 2023-11-16T23:59:19.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 msgr2=0x7fd55403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:19.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 0x7fd55403ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd560006fd0 tx=0x7fd560006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:19.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 msgr2=0x7fd56c196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c004f40 tx=0x7fd55c0057c0 comp rx=0 tx=0).stop 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 shutdown_connections 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd5540388c0 0x7fd55403ad70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 --2- 172.21.15.6:0/2947818731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd56c0ffd00 0x7fd56c196ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 >> 172.21.15.6:0/2947818731 conn(0x7fd56c0fb2b0 msgr2=0x7fd56c0fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 shutdown_connections 2023-11-16T23:59:19.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:19.092+0000 7fd570ebe700 1 -- 172.21.15.6:0/2947818731 wait complete. 2023-11-16T23:59:19.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi087 2023-11-16T23:59:19.666 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:59:19.666 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/etc/ceph/ceph.conf 2023-11-16T23:59:19.687 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-16T23:59:19.687 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:19.746 INFO:tasks.cephadm:Adding host smithi087 to orchestrator... 2023-11-16T23:59:19.747 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi087 2023-11-16T23:59:20.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:20.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:20.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:20.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:20.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:20.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-16T23:59:20.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-16T23:59:20.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:20 smithi006 ceph-mon[97840]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:20.630 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:21.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:21 smithi006 ceph-mon[97840]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:21.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:21 smithi006 ceph-mon[97840]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 -- 172.21.15.6:0/682288105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 msgr2=0x7f56b8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 --2- 172.21.15.6:0/682288105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b8100360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009910 tx=0x7f56b4009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 -- 172.21.15.6:0/682288105 shutdown_connections 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 --2- 172.21.15.6:0/682288105 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b8100360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 -- 172.21.15.6:0/682288105 >> 172.21.15.6:0/682288105 conn(0x7f56b80fb850 msgr2=0x7f56b80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:22.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 -- 172.21.15.6:0/682288105 shutdown_connections 2023-11-16T23:59:22.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.346+0000 7f56bfc44700 1 -- 172.21.15.6:0/682288105 wait complete. 2023-11-16T23:59:22.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.347+0000 7f56bfc44700 1 Processor -- start 2023-11-16T23:59:22.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.347+0000 7f56bfc44700 1 -- start start 2023-11-16T23:59:22.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bfc44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:22.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bfc44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8195700 con 0x7f56b80fff90 2023-11-16T23:59:22.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bd9e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:22.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bd9e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54076/0 (socket says 172.21.15.6:54076) 2023-11-16T23:59:22.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bd9e0700 1 -- 172.21.15.6:0/1401787860 learned_addr learned my addr 172.21.15.6:0/1401787860 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:22.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.348+0000 7f56bd9e0700 1 -- 172.21.15.6:0/1401787860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40095f0 con 0x7f56b80fff90 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56bd9e0700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56b401d070 con 0x7f56b80fff90 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b400bc10 con 0x7f56b80fff90 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b810a170 con 0x7f56b80fff90 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56b4003910 con 0x7f56b80fff90 2023-11-16T23:59:22.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.349+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b8109fe0 con 0x7f56b80fff90 2023-11-16T23:59:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.350+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f56b400bd80 con 0x7f56b80fff90 2023-11-16T23:59:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.350+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b810a300 con 0x7f56b80fff90 2023-11-16T23:59:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.350+0000 7f56aeffd700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 0x7f56a403adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.351+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f56b404c560 con 0x7f56b80fff90 2023-11-16T23:59:22.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.351+0000 7f56bd1df700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 0x7f56a403adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:22.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.351+0000 7f56bd1df700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 0x7f56a403adc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:22.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.356+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f56b4029360 con 0x7f56b80fff90 2023-11-16T23:59:22.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:22.566+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi087", "target": ["mon-mgr", ""]}) v1 -- 0x7f56b81080d0 con 0x7f56a4038910 2023-11-16T23:59:22.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:22.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:22.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:22.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:22 smithi006 ceph-mon[97840]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:24.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:23 smithi006 ceph-mon[97840]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.709+0000 7f56aeffd700 1 -- 172.21.15.6:0/1401787860 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f56b81080d0 con 0x7f56a4038910 2023-11-16T23:59:24.711 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi087' with addr '172.21.15.87' 2023-11-16T23:59:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 msgr2=0x7f56a403adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 0x7f56a403adc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 msgr2=0x7f56b81951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b4004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 shutdown_connections 2023-11-16T23:59:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f56a4038910 0x7f56a403adc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 --2- 172.21.15.6:0/1401787860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b80fff90 0x7f56b81951c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 >> 172.21.15.6:0/1401787860 conn(0x7f56b80fb850 msgr2=0x7f56b8104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:24.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 shutdown_connections 2023-11-16T23:59:24.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:24.712+0000 7f56bfc44700 1 -- 172.21.15.6:0/1401787860 wait complete. 2023-11-16T23:59:25.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:24 smithi006 ceph-mon[97840]: Deploying cephadm binary to smithi087 2023-11-16T23:59:25.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:24 smithi006 ceph-mon[97840]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:25.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:24 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:25.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:24 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:25.322 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T23:59:26.177 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:26.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:25 smithi006 ceph-mon[97840]: Added host smithi087 2023-11-16T23:59:26.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:25 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:26 smithi006 ceph-mon[97840]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:27.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 -- 172.21.15.6:0/917657980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 msgr2=0x7fde80100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 --2- 172.21.15.6:0/917657980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde80100360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fde70009910 tx=0x7fde70009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 -- 172.21.15.6:0/917657980 shutdown_connections 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 --2- 172.21.15.6:0/917657980 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde80100360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 -- 172.21.15.6:0/917657980 >> 172.21.15.6:0/917657980 conn(0x7fde800fb850 msgr2=0x7fde800fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 -- 172.21.15.6:0/917657980 shutdown_connections 2023-11-16T23:59:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.938+0000 7fde85f0c700 1 -- 172.21.15.6:0/917657980 wait complete. 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.939+0000 7fde85f0c700 1 Processor -- start 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.939+0000 7fde85f0c700 1 -- start start 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde85f0c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde85f0c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801978e0 con 0x7fde800fff90 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde7f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:27.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde7f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54106/0 (socket says 172.21.15.6:54106) 2023-11-16T23:59:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde7f7fe700 1 -- 172.21.15.6:0/234315005 learned_addr learned my addr 172.21.15.6:0/234315005 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde7f7fe700 1 -- 172.21.15.6:0/234315005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700095f0 con 0x7fde800fff90 2023-11-16T23:59:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.940+0000 7fde7f7fe700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde70006010 tx=0x7fde70004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.941+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde7001d070 con 0x7fde800fff90 2023-11-16T23:59:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.941+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7000bc10 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.941+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde70003910 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.941+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8010a170 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.941+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80109fe0 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.942+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fde7000bd80 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.942+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8010a300 con 0x7fde800fff90 2023-11-16T23:59:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.942+0000 7fde7cff9700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 0x7fde6803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.942+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fde7004c5a0 con 0x7fde800fff90 2023-11-16T23:59:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.943+0000 7fde7effd700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 0x7fde6803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.943+0000 7fde7effd700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 0x7fde6803ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:27.947+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fde70029360 con 0x7fde800fff90 2023-11-16T23:59:28.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.158+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde80197ee0 con 0x7fde680388c0 2023-11-16T23:59:28.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.159+0000 7fde7cff9700 1 -- 172.21.15.6:0/234315005 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7fde80197ee0 con 0x7fde680388c0 2023-11-16T23:59:28.161 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:28.161 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}] 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.161+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 msgr2=0x7fde6803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.161+0000 7fde85f0c700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 0x7fde6803ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 msgr2=0x7fde801973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde70006010 tx=0x7fde70004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 shutdown_connections 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fde680388c0 0x7fde6803ad70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 --2- 172.21.15.6:0/234315005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde800fff90 0x7fde801973a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 >> 172.21.15.6:0/234315005 conn(0x7fde800fb850 msgr2=0x7fde801038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 shutdown_connections 2023-11-16T23:59:28.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:28.162+0000 7fde85f0c700 1 -- 172.21.15.6:0/234315005 wait complete. 2023-11-16T23:59:28.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:28.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:28 smithi006 ceph-mon[97840]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:28.664 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2023-11-16T23:59:28.664 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:59:28.664 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2023-11-16T23:59:28.685 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-16T23:59:28.685 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:28.745 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2023-11-16T23:59:28.745 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi186 2023-11-16T23:59:29.602 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:29.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:29 smithi006 ceph-mon[97840]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:30.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:30.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:31.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.270+0000 7f1bbf544700 1 -- 172.21.15.6:0/741167407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 msgr2=0x7f1bb8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:31.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.270+0000 7f1bbf544700 1 --2- 172.21.15.6:0/741167407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8100360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009910 tx=0x7f1bb4009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:31.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.270+0000 7f1bbf544700 1 -- 172.21.15.6:0/741167407 shutdown_connections 2023-11-16T23:59:31.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.270+0000 7f1bbf544700 1 --2- 172.21.15.6:0/741167407 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8100360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:31.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.270+0000 7f1bbf544700 1 -- 172.21.15.6:0/741167407 >> 172.21.15.6:0/741167407 conn(0x7f1bb80fb850 msgr2=0x7f1bb80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:31.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.271+0000 7f1bbf544700 1 -- 172.21.15.6:0/741167407 shutdown_connections 2023-11-16T23:59:31.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.271+0000 7f1bbf544700 1 -- 172.21.15.6:0/741167407 wait complete. 2023-11-16T23:59:31.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.272+0000 7f1bbf544700 1 Processor -- start 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.272+0000 7f1bbf544700 1 -- start start 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbf544700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbf544700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8074d70 con 0x7f1bb80fff90 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbd2e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbd2e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35874/0 (socket says 172.21.15.6:35874) 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbd2e0700 1 -- 172.21.15.6:0/1692684878 learned_addr learned my addr 172.21.15.6:0/1692684878 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:31.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.273+0000 7f1bbd2e0700 1 -- 172.21.15.6:0/1692684878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bb80fff90 2023-11-16T23:59:31.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bbd2e0700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006010 tx=0x7f1bb4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:31.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bb401d070 con 0x7f1bb80fff90 2023-11-16T23:59:31.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb400bc10 con 0x7f1bb80fff90 2023-11-16T23:59:31.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bb4003910 con 0x7f1bb80fff90 2023-11-16T23:59:31.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb810a170 con 0x7f1bb80fff90 2023-11-16T23:59:31.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.274+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb8109fe0 con 0x7f1bb80fff90 2023-11-16T23:59:31.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.275+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f1bb400bd80 con 0x7f1bb80fff90 2023-11-16T23:59:31.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.275+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb810a300 con 0x7f1bb80fff90 2023-11-16T23:59:31.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.276+0000 7f1bae7fc700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 0x7f1ba403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:31.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.276+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1bb404c560 con 0x7f1bb80fff90 2023-11-16T23:59:31.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.276+0000 7f1bbcadf700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 0x7f1ba403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:31.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.276+0000 7f1bbcadf700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 0x7f1ba403ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8006fd0 tx=0x7f1ba8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:31.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.281+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bb4029360 con 0x7f1bb80fff90 2023-11-16T23:59:31.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:31.493+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb8074320 con 0x7f1ba40388c0 2023-11-16T23:59:32.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:31 smithi006 ceph-mon[97840]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:31 smithi006 ceph-mon[97840]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:31 smithi006 ceph-mon[97840]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:33.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:32 smithi006 ceph-mon[97840]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:33.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:32 smithi006 ceph-mon[97840]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:33.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.684+0000 7f1bae7fc700 1 -- 172.21.15.6:0/1692684878 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1bb8074320 con 0x7f1ba40388c0 2023-11-16T23:59:33.686 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi186' with addr '172.21.15.186' 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 msgr2=0x7f1ba403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 0x7f1ba403ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8006fd0 tx=0x7f1ba8006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 msgr2=0x7f1bb8072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006010 tx=0x7f1bb4004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 shutdown_connections 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1ba40388c0 0x7f1ba403ad70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 --2- 172.21.15.6:0/1692684878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bb80fff90 0x7f1bb8072df0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 >> 172.21.15.6:0/1692684878 conn(0x7f1bb80fb850 msgr2=0x7f1bb8104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 shutdown_connections 2023-11-16T23:59:33.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:33.687+0000 7f1bbf544700 1 -- 172.21.15.6:0/1692684878 wait complete. 2023-11-16T23:59:33.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:33 smithi006 ceph-mon[97840]: Deploying cephadm binary to smithi186 2023-11-16T23:59:33.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:33.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:34.198 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T23:59:35.057 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:34 smithi006 ceph-mon[97840]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:34 smithi006 ceph-mon[97840]: Added host smithi186 2023-11-16T23:59:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:35.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:35 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:36.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 -- 172.21.15.6:0/1635609048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 msgr2=0x7f60f4100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:36.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 --2- 172.21.15.6:0/1635609048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f4100360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009910 tx=0x7f60e8009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:36.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 -- 172.21.15.6:0/1635609048 shutdown_connections 2023-11-16T23:59:36.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 --2- 172.21.15.6:0/1635609048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f4100360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:36.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 -- 172.21.15.6:0/1635609048 >> 172.21.15.6:0/1635609048 conn(0x7f60f40fb850 msgr2=0x7f60f40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:36.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 -- 172.21.15.6:0/1635609048 shutdown_connections 2023-11-16T23:59:36.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.746+0000 7f60fb06a700 1 -- 172.21.15.6:0/1635609048 wait complete. 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.747+0000 7f60fb06a700 1 Processor -- start 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60fb06a700 1 -- start start 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60fb06a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60fb06a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4195700 con 0x7f60f40fff90 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60f8e06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60f8e06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35886/0 (socket says 172.21.15.6:35886) 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.748+0000 7f60f8e06700 1 -- 172.21.15.6:0/2454785006 learned_addr learned my addr 172.21.15.6:0/2454785006 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:36.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60f8e06700 1 -- 172.21.15.6:0/2454785006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80095f0 con 0x7f60f40fff90 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60f8e06700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e801d070 con 0x7f60f40fff90 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60e800bc10 con 0x7f60f40fff90 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f410a170 con 0x7f60f40fff90 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e8003910 con 0x7f60f40fff90 2023-11-16T23:59:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.749+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f4109fe0 con 0x7f60f40fff90 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.750+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f60e8003b80 con 0x7f60f40fff90 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.750+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f410a300 con 0x7f60f40fff90 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.751+0000 7f60f1ffb700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 0x7f60e403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.751+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60e804c540 con 0x7f60f40fff90 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.751+0000 7f60f3fff700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 0x7f60e403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.752+0000 7f60f3fff700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 0x7f60e403ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60e0006fd0 tx=0x7f60e0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:36.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.756+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f60e800ba70 con 0x7f60f40fff90 2023-11-16T23:59:36.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.971+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60f4195d00 con 0x7f60e40388c0 2023-11-16T23:59:36.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.972+0000 7f60f1ffb700 1 -- 172.21.15.6:0/2454785006 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f60f4195d00 con 0x7f60e40388c0 2023-11-16T23:59:36.974 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-16T23:59:36.974 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2023-11-16T23:59:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 msgr2=0x7f60e403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 0x7f60e403ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60e0006fd0 tx=0x7f60e0006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 msgr2=0x7f60f41951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:36.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 shutdown_connections 2023-11-16T23:59:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f60e40388c0 0x7f60e403ad70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 --2- 172.21.15.6:0/2454785006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60f40fff90 0x7f60f41951c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.974+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 >> 172.21.15.6:0/2454785006 conn(0x7f60f40fb850 msgr2=0x7f60f4104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.975+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 shutdown_connections 2023-11-16T23:59:36.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:36.975+0000 7f60fb06a700 1 -- 172.21.15.6:0/2454785006 wait complete. 2023-11-16T23:59:37.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:36 smithi006 ceph-mon[97840]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:37.147 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:37.537 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-16T23:59:37.537 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd crush tunables default 2023-11-16T23:59:38.399 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T23:59:38.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:38 smithi006 ceph-mon[97840]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:38.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:38 smithi006 ceph-mon[97840]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 -- 172.21.15.6:0/2072152510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 msgr2=0x7fcfd0073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/2072152510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0073200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009910 tx=0x7fcfcc009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 -- 172.21.15.6:0/2072152510 shutdown_connections 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/2072152510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0073200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 -- 172.21.15.6:0/2072152510 >> 172.21.15.6:0/2072152510 conn(0x7fcfd00fb850 msgr2=0x7fcfd00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 -- 172.21.15.6:0/2072152510 shutdown_connections 2023-11-16T23:59:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.078+0000 7fcfd80ae700 1 -- 172.21.15.6:0/2072152510 wait complete. 2023-11-16T23:59:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.080+0000 7fcfd80ae700 1 Processor -- start 2023-11-16T23:59:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.080+0000 7fcfd80ae700 1 -- start start 2023-11-16T23:59:40.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd80ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:40.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd80ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0197970 con 0x7fcfd0072e30 2023-11-16T23:59:40.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd5e4a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:40.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd5e4a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57000/0 (socket says 172.21.15.6:57000) 2023-11-16T23:59:40.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd5e4a700 1 -- 172.21.15.6:0/1383276402 learned_addr learned my addr 172.21.15.6:0/1383276402 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-16T23:59:40.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd5e4a700 1 -- 172.21.15.6:0/1383276402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0095f0 con 0x7fcfd0072e30 2023-11-16T23:59:40.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.081+0000 7fcfd5e4a700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc006010 tx=0x7fcfcc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:40.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.082+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfcc01d070 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.082+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfcc0044a0 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.082+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd010c380 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.082+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfcc003a70 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.082+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd010c1f0 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.083+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fcfcc003cb0 con 0x7fcfd0072e30 2023-11-16T23:59:40.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.083+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd010c510 con 0x7fcfd0072e30 2023-11-16T23:59:40.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.084+0000 7fcfc6ffd700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:40.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.084+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfcc04c6b0 con 0x7fcfd0072e30 2023-11-16T23:59:40.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.084+0000 7fcfd5649700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:40.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.084+0000 7fcfd5649700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 0x7fcfbc03ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:40.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.089+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcfcc005c00 con 0x7fcfd0072e30 2023-11-16T23:59:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:40.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.301+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcfd0109590 con 0x7fcfd0072e30 2023-11-16T23:59:40.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.928+0000 7fcfc6ffd700 1 -- 172.21.15.6:0/1383276402 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcfcc026020 con 0x7fcfd0072e30 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 msgr2=0x7fcfbc03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 0x7fcfbc03ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 msgr2=0x7fcfd0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc006010 tx=0x7fcfcc004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 shutdown_connections 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 --2- 172.21.15.6:0/1383276402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfd0072e30 0x7fcfd0197430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 >> 172.21.15.6:0/1383276402 conn(0x7fcfd00fb850 msgr2=0x7fcfd01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 shutdown_connections 2023-11-16T23:59:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-16T23:59:40.931+0000 7fcfd80ae700 1 -- 172.21.15.6:0/1383276402 wait complete. 2023-11-16T23:59:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2023-11-16T23:59:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-16T23:59:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-16T23:59:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:40 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T23:59:41.492 INFO:tasks.cephadm:Adding mon.a on smithi006 2023-11-16T23:59:41.492 INFO:tasks.cephadm:Adding mon.b on smithi087 2023-11-16T23:59:41.492 INFO:tasks.cephadm:Adding mon.c on smithi186 2023-11-16T23:59:41.492 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch apply mon '3;smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c' 2023-11-16T23:59:42.160 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:42.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T23:59:42.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:41 smithi006 ceph-mon[97840]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:42.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:41 smithi006 ceph-mon[97840]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:43.410 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- 172.21.15.186:0/4197765771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 msgr2=0x7f95fc1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:43.413 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 --2- 172.21.15.186:0/4197765771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1011c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009910 tx=0x7f95ec009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:43.413 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- 172.21.15.186:0/4197765771 shutdown_connections 2023-11-16T23:59:43.413 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 --2- 172.21.15.186:0/4197765771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1011c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:43.413 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- 172.21.15.186:0/4197765771 >> 172.21.15.186:0/4197765771 conn(0x7f95fc074930 msgr2=0x7f95fc074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:43.413 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- 172.21.15.186:0/4197765771 shutdown_connections 2023-11-16T23:59:43.414 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- 172.21.15.186:0/4197765771 wait complete. 2023-11-16T23:59:43.414 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 Processor -- start 2023-11-16T23:59:43.415 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.398+0000 7f9602a09700 1 -- start start 2023-11-16T23:59:43.415 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f9602a09700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:43.415 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f9602a09700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc195700 con 0x7f95fc100df0 2023-11-16T23:59:43.416 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f95fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:43.416 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f95fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.186:41746/0 (socket says 172.21.15.186:41746) 2023-11-16T23:59:43.416 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f95fbfff700 1 -- 172.21.15.186:0/4022915003 learned_addr learned my addr 172.21.15.186:0/4022915003 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-16T23:59:43.417 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f95fbfff700 1 -- 172.21.15.186:0/4022915003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0095f0 con 0x7f95fc100df0 2023-11-16T23:59:43.417 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.399+0000 7f95fbfff700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95ec005850 tx=0x7f95ec004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.400+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ec01d070 con 0x7f95fc100df0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.400+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc10a190 con 0x7f95fc100df0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.400+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc10a000 con 0x7f95fc100df0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.400+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95ec004370 con 0x7f95fc100df0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.400+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95ec003ab0 con 0x7f95fc100df0 2023-11-16T23:59:43.418 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.401+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc10a320 con 0x7f95fc100df0 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.401+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f95ec003c10 con 0x7f95fc100df0 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.401+0000 7f95f97fa700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 0x7f95e403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.401+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f95ec04c540 con 0x7f95fc100df0 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.401+0000 7f95fb7fe700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 0x7f95e403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.402+0000 7f95fb7fe700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 0x7f95e403ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006fd0 tx=0x7f95f0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:43.419 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.407+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95ec015760 con 0x7f95fc100df0 2023-11-16T23:59:43.599 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.599+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f95fc1073e0 con 0x7f95e4038870 2023-11-16T23:59:43.606 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.606+0000 7f95f97fa700 1 -- 172.21.15.186:0/4022915003 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f95fc1073e0 con 0x7f95e4038870 2023-11-16T23:59:43.606 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.607+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 msgr2=0x7f95e403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 0x7f95e403ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006fd0 tx=0x7f95f0006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 msgr2=0x7f95fc1951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95ec005850 tx=0x7f95ec004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 shutdown_connections 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f95e4038870 0x7f95e403ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 --2- 172.21.15.186:0/4022915003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95fc100df0 0x7f95fc1951c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:43.608 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 >> 172.21.15.186:0/4022915003 conn(0x7f95fc074930 msgr2=0x7f95fc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:43.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 shutdown_connections 2023-11-16T23:59:43.609 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:43.608+0000 7f9602a09700 1 -- 172.21.15.186:0/4022915003 wait complete. 2023-11-16T23:59:44.034 DEBUG:teuthology.orchestra.run.smithi087:mon.b> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.b.service 2023-11-16T23:59:44.037 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.c.service 2023-11-16T23:59:44.039 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T23:59:44.040 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T23:59:44.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-16T23:59:44.073 INFO:journalctl@ceph.mon.b.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-16T23:59:44.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:44.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c;count:3 2023-11-16T23:59:44.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:44.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:44.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:44.977 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:44.977 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: Deploying daemon mon.c on smithi186 2023-11-16T23:59:45.001 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:44 smithi006 ceph-mon[97840]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:45.059 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 -- 172.21.15.186:0/1426182136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 msgr2=0x7f117c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 --2- 172.21.15.186:0/1426182136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c0ff2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1178009910 tx=0x7f1178009c20 comp rx=0 tx=0).stop 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 -- 172.21.15.186:0/1426182136 shutdown_connections 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 --2- 172.21.15.186:0/1426182136 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c0ff2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 -- 172.21.15.186:0/1426182136 >> 172.21.15.186:0/1426182136 conn(0x7f117c074930 msgr2=0x7f117c074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 -- 172.21.15.186:0/1426182136 shutdown_connections 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.847+0000 7f11839d4700 1 -- 172.21.15.186:0/1426182136 wait complete. 2023-11-16T23:59:46.849 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.848+0000 7f11839d4700 1 Processor -- start 2023-11-16T23:59:46.850 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.848+0000 7f11839d4700 1 -- start start 2023-11-16T23:59:46.850 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.848+0000 7f11839d4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:46.850 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.848+0000 7f11839d4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c195720 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.849+0000 7f1181770700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.849+0000 7f1181770700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.186:41752/0 (socket says 172.21.15.186:41752) 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.849+0000 7f1181770700 1 -- 172.21.15.186:0/2825303012 learned_addr learned my addr 172.21.15.186:0/2825303012 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.849+0000 7f1181770700 1 -- 172.21.15.186:0/2825303012 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780095f0 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f1181770700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1178006010 tx=0x7f1178004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f117801d070 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c109090 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117800bc10 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1178003910 con 0x7f117c0fef00 2023-11-16T23:59:46.851 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.850+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c108f00 con 0x7f117c0fef00 2023-11-16T23:59:46.852 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.851+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f1178003bd0 con 0x7f117c0fef00 2023-11-16T23:59:46.852 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.851+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c109220 con 0x7f117c0fef00 2023-11-16T23:59:46.852 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.851+0000 7f11727fc700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 0x7f116803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:46.852 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.851+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f117804d4f0 con 0x7f117c0fef00 2023-11-16T23:59:46.852 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.852+0000 7f1180f6f700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 0x7f116803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:46.853 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.853+0000 7f1180f6f700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 0x7f116803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f116c006fd0 tx=0x7f116c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:46.857 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:46.857+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f117802b440 con 0x7f117c0fef00 2023-11-16T23:59:46.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:46 smithi006 ceph-mon[97840]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:46.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:46 smithi186 systemd[1]: Starting Ceph mon.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:59:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.131+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f117c062120 con 0x7f117c0fef00 2023-11-16T23:59:47.132 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.131+0000 7f11727fc700 1 -- 172.21.15.186:0/2825303012 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f1178026070 con 0x7f117c0fef00 2023-11-16T23:59:47.132 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:59:47.132 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T23:57:56.492691Z","created":"2023-11-16T23:57:56.492691Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-16T23:59:47.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 msgr2=0x7f116803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:47.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 0x7f116803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f116c006fd0 tx=0x7f116c006e40 comp rx=0 tx=0).stop 2023-11-16T23:59:47.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 msgr2=0x7f117c1951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:47.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1178006010 tx=0x7f1178004ff0 comp rx=0 tx=0).stop 2023-11-16T23:59:47.134 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 shutdown_connections 2023-11-16T23:59:47.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1168038870 0x7f116803ad20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:47.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 --2- 172.21.15.186:0/2825303012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f117c0fef00 0x7f117c1951e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:47.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 >> 172.21.15.186:0/2825303012 conn(0x7f117c074930 msgr2=0x7f117c067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:47.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 shutdown_connections 2023-11-16T23:59:47.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:47.134+0000 7f11839d4700 1 -- 172.21.15.186:0/2825303012 wait complete. 2023-11-16T23:59:47.136 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:46 smithi186 podman[131425]: 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pidfile_write: ignore empty --pid-file 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 bash[131425]: cc7a1c1574c0ae8e6880ae82362cf1ef01e938addfee02764e344004ef74e078 2023-11-16T23:59:47.231 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: load: jerasure load: lrc 2023-11-16T23:59:47.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: RocksDB version: 7.9.2 2023-11-16T23:59:47.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Git sha 0 2023-11-16T23:59:47.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-16T23:59:47.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: DB SUMMARY 2023-11-16T23:59:47.612 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: DB Session ID: N9Q90RKFNP9HKMMWPJTZ 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: CURRENT file: CURRENT 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.error_if_exists: 0 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.create_if_missing: 0 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.paranoid_checks: 1 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.env: 0x555eae2fb720 2023-11-16T23:59:47.613 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.info_log: 0x555eb17813c0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.statistics: (nil) 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.use_fsync: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_log_file_size: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_fallocate: 1 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T23:59:47.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.use_direct_reads: 0 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.db_log_dir: 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.wal_dir: 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T23:59:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T23:59:47.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.write_buffer_manager: 0x555eb09f23c0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T23:59:47.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.unordered_write: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.row_cache: None 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.wal_filter: None 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.two_write_queues: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.wal_compression: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.atomic_flush: 0 2023-11-16T23:59:47.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.log_readahead_size: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_background_jobs: 2 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_background_compactions: -1 2023-11-16T23:59:47.619 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_subcompactions: 1 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_open_files: -1 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T23:59:47.620 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_background_flushes: -1 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Compression algorithms supported: 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kZSTD supported: 0 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kXpressCompression supported: 0 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kZlibCompression supported: 1 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kSnappyCompression supported: 1 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kLZ4Compression supported: 1 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: kBZip2Compression supported: 0 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T23:59:47.621 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.merge_operator: 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_filter: None 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555eb17814e0) 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks: 1 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_top_level_index_and_filter: 1 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_type: 0 2023-11-16T23:59:47.622 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_index_type: 0 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_shortening: 1 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: checksum: 4 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: no_block_cache: 0 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache: 0x555eb0a931f0 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_name: BinnedLRUCache 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_options: 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: capacity : 536870912 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_shard_bits : 4 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: strict_capacity_limit : 0 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: high_pri_pool_ratio: 0.000 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_compressed: (nil) 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: persistent_cache: (nil) 2023-11-16T23:59:47.623 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size: 4096 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size_deviation: 10 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_restart_interval: 16 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_block_restart_interval: 1 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: metadata_block_size: 4096 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: partition_filters: 0 2023-11-16T23:59:47.722 INFO:journalctl@ceph.mon.c.smithi186.stdout: use_delta_encoding: 1 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: filter_policy: bloomfilter 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: whole_key_filtering: 1 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: verify_compression: 0 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: read_amp_bytes_per_bit: 0 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: format_version: 5 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_index_compression: 1 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_align: 0 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: max_auto_readahead_size: 262144 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: prepopulate_block_cache: 0 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: initial_auto_readahead_size: 8192 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T23:59:47.723 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression: NoCompression 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.num_levels: 7 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T23:59:47.724 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T23:59:47.746 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T23:59:47.747 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T23:59:47.748 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T23:59:47.749 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.table_properties_collectors: 2023-11-16T23:59:47.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.inplace_update_support: 0 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.bloom_locality: 0 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.max_successive_merges: 0 2023-11-16T23:59:47.770 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.ttl: 2592000 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enable_blob_files: false 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.min_blob_size: 0 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T23:59:47.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5f0f0eff-eb7b-4408-a773-f4cfc323f3f2 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179187233043, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179187233489, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700179187, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5f0f0eff-eb7b-4408-a773-f4cfc323f3f2", "db_session_id": "N9Q90RKFNP9HKMMWPJTZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179187233557, "job": 1, "event": "recovery_finished"} 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T23:59:47.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555eb0b30000 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: DB pointer 0x555eb0b1c000 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** DB Stats ** 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2023-11-16T23:59:47.773 INFO:journalctl@ceph.mon.c.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.774 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache BinnedLRUCache@0x555eb0a931f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T23:59:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: starting mon.c rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(???) e0 preinit fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).mds e1 new map 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: e1 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: legacy client fscid: -1 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout: No filesystems configured 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T23:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:47.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:47.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:47.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-16T23:59:47.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-16T23:59:47.783 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:47.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Deploying cephadm binary to smithi186 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Added host smithi186 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:47.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:47.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T23:59:47.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T23:59:47.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c;count:3 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:47.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: Deploying daemon mon.c on smithi186 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expand map: {default=false} 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta from 'false' to 'false' 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expanded map: {default=false} 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expand map: {default=info} 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta from 'info' to 'info' 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expanded map: {default=info} 2023-11-16T23:59:47.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expand map: {default=daemon} 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expand map: {default=debug} 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 ceph-mon[131487]: expand_channel_meta expanded map: {default=debug} 2023-11-16T23:59:47.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:47 smithi186 systemd[1]: Started Ceph mon.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:59:48.885 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T23:59:48.885 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T23:59:49.539 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T23:59:52.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: mon.a calling monitor election 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: mon.c calling monitor election 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: fsmap 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: mgrmap e13: x(active, since 50s) 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: overall HEALTH_OK 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:52.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:52.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: mon.a calling monitor election 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: mon.c calling monitor election 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T23:59:52.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: fsmap 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: mgrmap e13: x(active, since 50s) 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: overall HEALTH_OK 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:52.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.704+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3660769792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fcc0035c0 msgr2=0x7f5fcc005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.704+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3660769792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fcc0035c0 0x7f5fcc005a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4076150 tx=0x7f5fd800fe60 comp rx=0 tx=0).stop 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3660769792 shutdown_connections 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3660769792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fcc0035c0 0x7f5fcc005a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3660769792 >> 172.21.15.186:0/3660769792 conn(0x7f5fe4074c40 msgr2=0x7f5fe4075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3660769792 shutdown_connections 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3660769792 wait complete. 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.705+0000 7f5fe9d5e700 1 Processor -- start 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe9d5e700 1 -- start start 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe9d5e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:53.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe9d5e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 0x7f5fe4196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe9d5e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419f1b0 con 0x7f5fe4196680 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe9d5e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419f2f0 con 0x7f5fcc0035c0 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55540/0 (socket says 172.21.15.186:55540) 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 -- 172.21.15.186:0/3768173216 learned_addr learned my addr 172.21.15.186:0/3768173216 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 -- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 msgr2=0x7f5fe4196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 -- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 msgr2=0x7f5fe4196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.706+0000 7f5fe37fe700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe37fe700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T23:59:53.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe2ffd700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 0x7f5fe4196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:53.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe2ffd700 1 -- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 msgr2=0x7f5fe4196140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe2ffd700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe2ffd700 1 -- 172.21.15.186:0/3768173216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd800f8b0 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.707+0000 7f5fe2ffd700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 0x7f5fe4196af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400b4d0 tx=0x7f5fd400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.708+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f5fd4011630 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.708+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd4011c50 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.708+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe4106c50 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.708+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f5fd4010d90 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.708+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe4106ac0 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.709+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f5fd401a400 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.709+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4106de0 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.709+0000 7f5fe0ff9700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 0x7f5fd003ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.709+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5fd404ad90 con 0x7f5fe4196680 2023-11-16T23:59:53.710 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.709+0000 7f5fe37fe700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 0x7f5fd003ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T23:59:53.711 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.710+0000 7f5fe37fe700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 0x7f5fd003ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005b40 tx=0x7f5fd8005ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T23:59:53.715 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.715+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5fd4050020 con 0x7f5fe4196680 2023-11-16T23:59:53.982 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.981+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5fe4062120 con 0x7f5fe4196680 2023-11-16T23:59:53.983 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.982+0000 7f5fe0ff9700 1 -- 172.21.15.186:0/3768173216 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f5fd4017680 con 0x7f5fe4196680 2023-11-16T23:59:53.983 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-16T23:59:53.983 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T23:59:47.258945Z","created":"2023-11-16T23:57:56.492691Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-16T23:59:53.985 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 msgr2=0x7f5fd003ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:53.985 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 0x7f5fd003ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005b40 tx=0x7f5fd8005ad0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.985 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 msgr2=0x7f5fe4196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 0x7f5fe4196af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400b4d0 tx=0x7f5fd400b890 comp rx=0 tx=0).stop 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 shutdown_connections 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5fd00386b0 0x7f5fd003ab60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fcc0035c0 0x7f5fe4196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 --2- 172.21.15.186:0/3768173216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5fe4196680 0x7f5fe4196af0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 >> 172.21.15.186:0/3768173216 conn(0x7f5fe4074c40 msgr2=0x7f5fe418cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T23:59:53.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 shutdown_connections 2023-11-16T23:59:54.047 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-16T23:59:53.984+0000 7f5fe9d5e700 1 -- 172.21.15.186:0/3768173216 wait complete. 2023-11-16T23:59:54.047 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2023-11-16T23:59:54.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:53 smithi186 ceph-mon[131487]: Deploying daemon mon.b on smithi087 2023-11-16T23:59:54.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:54.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:53 smithi006 ceph-mon[97840]: Deploying daemon mon.b on smithi087 2023-11-16T23:59:54.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:54.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 16 23:59:54 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-16T23:59:54.258+0000 7fdbcc94a700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-16T23:59:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:54 smithi087 systemd[1]: Starting Ceph mon.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-16T23:59:55.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:54 smithi186 ceph-mon[131487]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 16 23:59:54 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/3768173216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T23:59:55.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:54 smithi006 ceph-mon[97840]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 16 23:59:54 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/3768173216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T23:59:55.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 podman[131142]: 2023-11-16T23:59:55.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 bash[131142]: 0f17a802242f0ed4fefe9a2762743903b0a3ccee1cfb21db31d4763cff86a8a1 2023-11-16T23:59:55.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T23:59:55.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-16T23:59:55.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pidfile_write: ignore empty --pid-file 2023-11-16T23:59:55.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: load: jerasure load: lrc 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: RocksDB version: 7.9.2 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Git sha 0 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: DB SUMMARY 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: DB Session ID: B4RAY6HORPE79HR6WWJ6 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: CURRENT file: CURRENT 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.error_if_exists: 0 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.create_if_missing: 0 2023-11-16T23:59:55.404 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.paranoid_checks: 1 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.env: 0x5569300a9720 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T23:59:55.405 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.info_log: 0x556931ca73c0 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.statistics: (nil) 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.use_fsync: 0 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_log_file_size: 0 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T23:59:55.407 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T23:59:55.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T23:59:55.408 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_fallocate: 1 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.use_direct_reads: 0 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T23:59:55.490 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.db_log_dir: 2023-11-16T23:59:55.491 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.wal_dir: 2023-11-16T23:59:55.491 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T23:59:55.491 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T23:59:55.491 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.write_buffer_manager: 0x556930f183c0 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T23:59:55.515 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.unordered_write: 0 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.row_cache: None 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.wal_filter: None 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T23:59:55.516 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.two_write_queues: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.wal_compression: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.atomic_flush: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.log_readahead_size: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T23:59:55.517 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_background_jobs: 2 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_background_compactions: -1 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_subcompactions: 1 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T23:59:55.518 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T23:59:55.519 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T23:59:55.519 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T23:59:55.519 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_open_files: -1 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_background_flushes: -1 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Compression algorithms supported: 2023-11-16T23:59:55.546 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kZSTD supported: 0 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kXpressCompression supported: 0 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kZlibCompression supported: 1 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kSnappyCompression supported: 1 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kLZ4Compression supported: 1 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: kBZip2Compression supported: 0 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T23:59:55.547 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.merge_operator: 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_filter: None 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556931ca74e0) 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks: 1 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: pin_top_level_index_and_filter: 1 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_type: 0 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_index_type: 0 2023-11-16T23:59:55.548 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_shortening: 1 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: checksum: 4 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: no_block_cache: 0 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache: 0x556930fb91f0 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_name: BinnedLRUCache 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_options: 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: capacity : 536870912 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: num_shard_bits : 4 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: strict_capacity_limit : 0 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: high_pri_pool_ratio: 0.000 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_cache_compressed: (nil) 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: persistent_cache: (nil) 2023-11-16T23:59:55.549 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size: 4096 2023-11-16T23:59:55.550 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_size_deviation: 10 2023-11-16T23:59:55.550 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_restart_interval: 16 2023-11-16T23:59:55.550 INFO:journalctl@ceph.mon.b.smithi087.stdout: index_block_restart_interval: 1 2023-11-16T23:59:55.550 INFO:journalctl@ceph.mon.b.smithi087.stdout: metadata_block_size: 4096 2023-11-16T23:59:55.643 INFO:journalctl@ceph.mon.b.smithi087.stdout: partition_filters: 0 2023-11-16T23:59:55.643 INFO:journalctl@ceph.mon.b.smithi087.stdout: use_delta_encoding: 1 2023-11-16T23:59:55.643 INFO:journalctl@ceph.mon.b.smithi087.stdout: filter_policy: bloomfilter 2023-11-16T23:59:55.669 INFO:journalctl@ceph.mon.b.smithi087.stdout: whole_key_filtering: 1 2023-11-16T23:59:55.669 INFO:journalctl@ceph.mon.b.smithi087.stdout: verify_compression: 0 2023-11-16T23:59:55.672 INFO:journalctl@ceph.mon.b.smithi087.stdout: read_amp_bytes_per_bit: 0 2023-11-16T23:59:55.672 INFO:journalctl@ceph.mon.b.smithi087.stdout: format_version: 5 2023-11-16T23:59:55.672 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_index_compression: 1 2023-11-16T23:59:55.672 INFO:journalctl@ceph.mon.b.smithi087.stdout: block_align: 0 2023-11-16T23:59:55.685 INFO:journalctl@ceph.mon.b.smithi087.stdout: max_auto_readahead_size: 262144 2023-11-16T23:59:55.685 INFO:journalctl@ceph.mon.b.smithi087.stdout: prepopulate_block_cache: 0 2023-11-16T23:59:55.685 INFO:journalctl@ceph.mon.b.smithi087.stdout: initial_auto_readahead_size: 8192 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression: NoCompression 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.num_levels: 7 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T23:59:55.686 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T23:59:55.687 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T23:59:55.688 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T23:59:55.689 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T23:59:55.690 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.table_properties_collectors: 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.inplace_update_support: 0 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T23:59:55.691 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.bloom_locality: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.max_successive_merges: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.ttl: 2592000 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T23:59:55.692 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enable_blob_files: false 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.min_blob_size: 0 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d22af9ca-22db-42dd-83fc-3245f53e5d68 2023-11-16T23:59:55.693 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179195214958, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179195215372, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700179195, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d22af9ca-22db-42dd-83fc-3245f53e5d68", "db_session_id": "B4RAY6HORPE79HR6WWJ6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700179195215438, "job": 1, "event": "recovery_finished"} 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556931056000 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: DB pointer 0x556931042000 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T23:59:55.694 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** DB Stats ** 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T23:59:55.695 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** Compaction Stats [default] ** 2023-11-16T23:59:55.717 INFO:journalctl@ceph.mon.b.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T23:59:55.718 INFO:journalctl@ceph.mon.b.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T23:59:55.718 INFO:journalctl@ceph.mon.b.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T23:59:55.719 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.719 INFO:journalctl@ceph.mon.b.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T23:59:55.719 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.719 INFO:journalctl@ceph.mon.b.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Block cache BinnedLRUCache@0x556930fb91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T23:59:55.720 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: using public_addr v2:172.21.15.87:0/0 -> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: starting mon.b rank -1 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(???) e0 preinit fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).mds e1 new map 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: e1 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: legacy client fscid: -1 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: 2023-11-16T23:59:55.721 INFO:journalctl@ceph.mon.b.smithi087.stdout: No filesystems configured 2023-11-16T23:59:55.722 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T23:59:55.722 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T23:59:55.722 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:55.785 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.786 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Deploying cephadm binary to smithi186 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Added host smithi186 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.788 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1383276402' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.789 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T23:59:55.790 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c;count:3 2023-11-16T23:59:55.790 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.790 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T23:59:55.790 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:55.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T23:59:55.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Deploying daemon mon.c on smithi186 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T23:59:55.800 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.a calling monitor election 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.c calling monitor election 2023-11-16T23:59:55.801 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: fsmap 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mgrmap e13: x(active, since 50s) 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: overall HEALTH_OK 2023-11-16T23:59:55.802 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.803 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.804 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: Deploying daemon mon.b on smithi087 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/3768173216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T23:59:55.805 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expand map: {default=false} 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta from 'false' to 'false' 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expanded map: {default=false} 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expand map: {default=info} 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta from 'info' to 'info' 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expanded map: {default=info} 2023-11-16T23:59:55.806 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expand map: {default=daemon} 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expand map: {default=debug} 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 ceph-mon[131182]: expand_channel_meta expanded map: {default=debug} 2023-11-16T23:59:55.807 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 16 23:59:55 smithi087 systemd[1]: Started Ceph mon.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-16T23:59:55.807 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T23:59:55.807 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T23:59:56.461 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: mon.a calling monitor election 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: mon.c calling monitor election 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: mon.b calling monitor election 2023-11-17T00:00:00.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: fsmap 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: osdmap e4: 0 total, 0 up, 0 in 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: mgrmap e13: x(active, since 58s) 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: overall HEALTH_OK 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: mon.a calling monitor election 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: mon.c calling monitor election 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: mon.b calling monitor election 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: fsmap 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: osdmap e4: 0 total, 0 up, 0 in 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: mgrmap e13: x(active, since 58s) 2023-11-17T00:00:00.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: overall HEALTH_OK 2023-11-17T00:00:00.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.703 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 -- 172.21.15.186:0/1152224490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 msgr2=0x7fa5dc005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:00.703 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1152224490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5dc005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00ffda0 tx=0x7fa5d800feb0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 -- 172.21.15.186:0/1152224490 shutdown_connections 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1152224490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5dc005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 -- 172.21.15.186:0/1152224490 >> 172.21.15.186:0/1152224490 conn(0x7fa5f00fb990 msgr2=0x7fa5f00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 -- 172.21.15.186:0/1152224490 shutdown_connections 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.701+0000 7fa5f608e700 1 -- 172.21.15.186:0/1152224490 wait complete. 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 Processor -- start 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 -- start start 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5f0195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:00.704 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:00.706 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:00.706 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0195f70 con 0x7fa5dc0035c0 2023-11-17T00:00:00.706 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01a1fc0 con 0x7fa5f01958a0 2023-11-17T00:00:00.706 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.702+0000 7fa5f608e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01a2100 con 0x7fa5f019a250 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55562/0 (socket says 172.21.15.186:55562) 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 -- 172.21.15.186:0/1065682741 learned_addr learned my addr 172.21.15.186:0/1065682741 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 msgr2=0x7fa5f019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 msgr2=0x7fa5f019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5eeffd700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5effff700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5f0195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5eeffd700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 msgr2=0x7fa5f0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5eeffd700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 msgr2=0x7fa5f0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-17T00:00:00.707 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5eeffd700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5eeffd700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 msgr2=0x7fa5f019a6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 msgr2=0x7fa5f0199d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 -- 172.21.15.186:0/1065682741 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d800f900 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.703+0000 7fa5ef7fe700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5f0195360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00ffda0 tx=0x7fa5d80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.704+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80056f0 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.704+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f0107c70 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.704+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5d8004030 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.704+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8003ac0 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.704+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f0107ae0 con 0x7fa5dc0035c0 2023-11-17T00:00:00.708 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.705+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fa5d8003c20 con 0x7fa5dc0035c0 2023-11-17T00:00:00.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.705+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f0107e00 con 0x7fa5dc0035c0 2023-11-17T00:00:00.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.705+0000 7fa5ecff9700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 0x7fa5d003b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:00.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.705+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa5d804eb50 con 0x7fa5dc0035c0 2023-11-17T00:00:00.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.705+0000 7fa5eeffd700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 0x7fa5d003b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:00.709 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.706+0000 7fa5eeffd700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 0x7fa5d003b1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009850 tx=0x7fa5e0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:00.712 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.710+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa5d80109c0 con 0x7fa5dc0035c0 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: mon.a calling monitor election 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: mon.c calling monitor election 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: mon.b calling monitor election 2023-11-17T00:00:00.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: fsmap 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: osdmap e4: 0 total, 0 up, 0 in 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: mgrmap e13: x(active, since 58s) 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: overall HEALTH_OK 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:00.993 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.991+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa5f0062120 con 0x7fa5dc0035c0 2023-11-17T00:00:00.993 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.992+0000 7fa5ecff9700 1 -- 172.21.15.186:0/1065682741 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1268 (secure 0 0 0) 0x7fa5d80103d0 con 0x7fa5dc0035c0 2023-11-17T00:00:00.993 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-11-17T00:00:00.994 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T23:59:55.239601Z","created":"2023-11-16T23:57:56.492691Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-11-17T00:00:00.995 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 msgr2=0x7fa5d003b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:00.995 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 0x7fa5d003b1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009850 tx=0x7fa5e0008040 comp rx=0 tx=0).stop 2023-11-17T00:00:00.995 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 msgr2=0x7fa5f0195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5f0195360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00ffda0 tx=0x7fa5d80052d0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 shutdown_connections 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5d0038cf0 0x7fa5d003b1a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc0035c0 0x7fa5f0195360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5f019a250 0x7fa5f019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 --2- 172.21.15.186:0/1065682741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5f01958a0 0x7fa5f0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 >> 172.21.15.186:0/1065682741 conn(0x7fa5f00fb990 msgr2=0x7fa5f018d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 shutdown_connections 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:00.994+0000 7fa5f608e700 1 -- 172.21.15.186:0/1065682741 wait complete. 2023-11-17T00:00:00.996 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2023-11-17T00:00:01.502 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:01 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/1065682741' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:01.512 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-17T00:00:01.512 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph config generate-minimal-conf 2023-11-17T00:00:01.585 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:01 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/1065682741' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:01.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:01.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:01 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/1065682741' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-17T00:00:01.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:00:02.416 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:00:02.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:03.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.148 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:03.148 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:03.149 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.132 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:04.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 -- 172.21.15.6:0/3454284784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c0fff90 msgr2=0x7f2d7c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 --2- 172.21.15.6:0/3454284784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c100360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009910 tx=0x7f2d6c009c20 comp rx=0 tx=0).stop 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 -- 172.21.15.6:0/3454284784 shutdown_connections 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 --2- 172.21.15.6:0/3454284784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c100360 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 -- 172.21.15.6:0/3454284784 >> 172.21.15.6:0/3454284784 conn(0x7f2d7c0fb850 msgr2=0x7f2d7c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 -- 172.21.15.6:0/3454284784 shutdown_connections 2023-11-17T00:00:04.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.380+0000 7f2d82110700 1 -- 172.21.15.6:0/3454284784 wait complete. 2023-11-17T00:00:04.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.381+0000 7f2d82110700 1 Processor -- start 2023-11-17T00:00:04.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 -- start start 2023-11-17T00:00:04.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:04.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:04.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7c0738c0 0x7f2d7c1a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:04.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1a83d0 con 0x7f2d7c074b10 2023-11-17T00:00:04.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1a8540 con 0x7f2d7c0738c0 2023-11-17T00:00:04.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d82110700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1a8700 con 0x7f2d7c0fff90 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.382+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44008/0 (socket says 172.21.15.6:44008) 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 -- 172.21.15.6:0/4031258492 learned_addr learned my addr 172.21.15.6:0/4031258492 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7bfff700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7c0738c0 0x7f2d7c1a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:04.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 -- 172.21.15.6:0/4031258492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d7c0fff90 msgr2=0x7f2d7c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:04.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 -- 172.21.15.6:0/4031258492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7c0738c0 msgr2=0x7f2d7c1a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:04.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7c0738c0 0x7f2d7c1a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 -- 172.21.15.6:0/4031258492 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0095f0 con 0x7f2d7c074b10 2023-11-17T00:00:04.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.383+0000 7f2d7affd700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b780 tx=0x7f2d7000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:04.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.384+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70004230 con 0x7f2d7c074b10 2023-11-17T00:00:04.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.384+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d70004390 con 0x7f2d7c074b10 2023-11-17T00:00:04.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.384+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c10a170 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.384+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70011b90 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.384+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c109fe0 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.385+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c10a300 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.385+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f2d70011cf0 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.386+0000 7f2d78ff9700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 0x7f2d6403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.386+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d70049f50 con 0x7f2d7c074b10 2023-11-17T00:00:04.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.386+0000 7f2d7b7fe700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 0x7f2d6403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:04.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.387+0000 7f2d7b7fe700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 0x7f2d6403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009fd0 tx=0x7f2d6c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:04.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.391+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d70003bf0 con 0x7f2d7c074b10 2023-11-17T00:00:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.635+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2d7c1a92b0 con 0x7f2d7c074b10 2023-11-17T00:00:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.636+0000 7f2d78ff9700 1 -- 172.21.15.6:0/4031258492 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f2d70016030 con 0x7f2d7c074b10 2023-11-17T00:00:04.637 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-17T00:00:04.638 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2023-11-17T00:00:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-17T00:00:04.638 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 msgr2=0x7f2d6403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 0x7f2d6403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009fd0 tx=0x7f2d6c0058e0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 msgr2=0x7f2d7c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b780 tx=0x7f2d7000ba90 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 shutdown_connections 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2d64038d40 0x7f2d6403b1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d7c0738c0 0x7f2d7c1a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d7c0fff90 0x7f2d7c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 --2- 172.21.15.6:0/4031258492 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d7c074b10 0x7f2d7c073330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.638+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 >> 172.21.15.6:0/4031258492 conn(0x7f2d7c0fb850 msgr2=0x7f2d7c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.639+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 shutdown_connections 2023-11-17T00:00:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:04.639+0000 7f2d82110700 1 -- 172.21.15.6:0/4031258492 wait complete. 2023-11-17T00:00:04.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:04.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:04.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:04.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: Reconfiguring mon.a (unknown last config time)... 2023-11-17T00:00:04.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: Reconfiguring daemon mon.a on smithi006 2023-11-17T00:00:04.935 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:04 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4031258492' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: Reconfiguring mon.a (unknown last config time)... 2023-11-17T00:00:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: Reconfiguring daemon mon.a on smithi006 2023-11-17T00:00:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:04 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4031258492' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:05.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:00:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: Reconfiguring mon.a (unknown last config time)... 2023-11-17T00:00:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: Reconfiguring daemon mon.a on smithi006 2023-11-17T00:00:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:04 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4031258492' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:05.416 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-17T00:00:05.417 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:00:05.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2023-11-17T00:00:05.494 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:00:05.494 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:00:05.605 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-17T00:00:05.605 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.conf 2023-11-17T00:00:05.640 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-17T00:00:05.641 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:00:05.714 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-17T00:00:05.715 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2023-11-17T00:00:05.754 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-17T00:00:05.754 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:00:05.825 INFO:tasks.cephadm:Adding mgr.x on smithi006 2023-11-17T00:00:05.825 INFO:tasks.cephadm:Adding mgr.y on smithi087 2023-11-17T00:00:05.825 INFO:tasks.cephadm:Adding mgr.z on smithi186 2023-11-17T00:00:05.825 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch apply mgr '3;smithi006=x;smithi087=y;smithi186=z' 2023-11-17T00:00:06.508 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:00:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:07.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:07.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:07.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 -- 172.21.15.186:0/2396900473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e140fe4c0 msgr2=0x7f0e140fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:07.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/2396900473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fe8d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400b0a0 tx=0x7f0e0400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 -- 172.21.15.186:0/2396900473 shutdown_connections 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/2396900473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fe8d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 -- 172.21.15.186:0/2396900473 >> 172.21.15.186:0/2396900473 conn(0x7f0e140f9a70 msgr2=0x7f0e140fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 -- 172.21.15.186:0/2396900473 shutdown_connections 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.737+0000 7f0e1be8c700 1 -- 172.21.15.186:0/2396900473 wait complete. 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 Processor -- start 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 -- start start 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 0x7f0e1419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419f660 con 0x7f0e1419a2b0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419f7d0 con 0x7f0e140fe4c0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1be8c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1419f990 con 0x7f0e1419cbd0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55572/0 (socket says 172.21.15.186:55572) 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 -- 172.21.15.186:0/324132644 learned_addr learned my addr 172.21.15.186:0/324132644 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e19427700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 0x7f0e1419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e19c28700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 -- 172.21.15.186:0/324132644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e140fe4c0 msgr2=0x7f0e140fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:07.742 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 -- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 msgr2=0x7f0e1419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 0x7f0e1419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e1a429700 1 -- 172.21.15.186:0/324132644 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04009d00 con 0x7f0e1419cbd0 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.738+0000 7f0e19427700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 0x7f0e1419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e1a429700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b550 tx=0x7f0e0c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:07.743 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c011830 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e0c011e50 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14104340 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00f2e0 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.739+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e141041b0 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.740+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f0e0c00f440 con 0x7f0e1419cbd0 2023-11-17T00:00:07.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.740+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e141a0120 con 0x7f0e1419cbd0 2023-11-17T00:00:07.745 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.740+0000 7f0e12ffd700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 0x7f0dfc03b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:07.745 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.740+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0e0c04aff0 con 0x7f0e1419cbd0 2023-11-17T00:00:07.745 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.740+0000 7f0e19c28700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 0x7f0dfc03b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:07.745 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.741+0000 7f0e19c28700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 0x7f0dfc03b1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e04015040 tx=0x7f0e0400b7d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:07.746 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.745+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0e0c07d730 con 0x7f0e1419cbd0 2023-11-17T00:00:07.959 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.957+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi087=y;smithi186=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e14068270 con 0x7f0dfc038d00 2023-11-17T00:00:07.968 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.966+0000 7f0e12ffd700 1 -- 172.21.15.186:0/324132644 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0e14068270 con 0x7f0dfc038d00 2023-11-17T00:00:07.969 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 msgr2=0x7f0dfc03b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 0x7f0dfc03b1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e04015040 tx=0x7f0e0400b7d0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 msgr2=0x7f0e1419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b550 tx=0x7f0e0c00b910 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 shutdown_connections 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0dfc038d00 0x7f0dfc03b1b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e1419cbd0 0x7f0e1419efb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e140fe4c0 0x7f0e140fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 --2- 172.21.15.186:0/324132644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e1419a2b0 0x7f0e1419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.968+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 >> 172.21.15.186:0/324132644 conn(0x7f0e140f9a70 msgr2=0x7f0e140fb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:07.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.969+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 shutdown_connections 2023-11-17T00:00:07.971 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:00:07.969+0000 7f0e1be8c700 1 -- 172.21.15.186:0/324132644 wait complete. 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: Reconfiguring mon.b (monmap changed)... 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: Reconfiguring daemon mon.b on smithi087 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:08.208 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:07 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:08.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: Reconfiguring mon.b (monmap changed)... 2023-11-17T00:00:08.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: Reconfiguring daemon mon.b on smithi087 2023-11-17T00:00:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:07 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:08.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: Reconfiguring mon.b (monmap changed)... 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: Reconfiguring daemon mon.b on smithi087 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-17T00:00:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:07 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:08.656 DEBUG:teuthology.orchestra.run.smithi087:mgr.y> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.y.service 2023-11-17T00:00:08.658 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.z.service 2023-11-17T00:00:08.661 INFO:tasks.cephadm:Deploying OSDs... 2023-11-17T00:00:08.661 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:00:08.661 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2023-11-17T00:00:08.690 INFO:journalctl@ceph.mgr.z.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:00:08.693 INFO:journalctl@ceph.mgr.y.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:00:08.710 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-17T00:00:08.710 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2023-11-17T00:00:08.757 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-16 23:59:12.751043104 +0000 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-16 23:53:19.225216197 +0000 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-16 23:53:19.225216197 +0000 2023-11-17T00:00:08.758 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-16 23:53:19.196216500 +0000 2023-11-17T00:00:08.758 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-17T00:00:08.830 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-17T00:00:08.831 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-17T00:00:08.831 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000309645 s, 1.7 MB/s 2023-11-17T00:00:08.833 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-17T00:00:08.884 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2023-11-17T00:00:08.932 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-17T00:00:08.932 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:08.932 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-11-17T00:00:08.932 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:08.933 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:08.933 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-16 23:59:12.751043104 +0000 2023-11-17T00:00:08.933 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-16 23:53:20.059207478 +0000 2023-11-17T00:00:08.933 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-16 23:53:20.059207478 +0000 2023-11-17T00:00:08.933 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-16 23:53:20.033207750 +0000 2023-11-17T00:00:08.933 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-17T00:00:09.001 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-17T00:00:09.118 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-17T00:00:09.118 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000308157 s, 1.7 MB/s 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: Reconfiguring mon.c (monmap changed)... 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: Reconfiguring daemon mon.c on smithi186 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:09.119 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: Reconfiguring mon.c (monmap changed)... 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: Reconfiguring daemon mon.c on smithi186 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.120 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:09.121 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-17T00:00:09.168 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2023-11-17T00:00:09.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: Reconfiguring mon.c (monmap changed)... 2023-11-17T00:00:09.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: Reconfiguring daemon mon.c on smithi186 2023-11-17T00:00:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:09.257 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-17T00:00:09.398 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:09.422 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-16 23:59:12.751043104 +0000 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-16 23:53:20.918198497 +0000 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-16 23:53:20.918198497 +0000 2023-11-17T00:00:09.430 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-16 23:53:20.890198790 +0000 2023-11-17T00:00:09.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-17T00:00:09.501 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-17T00:00:09.501 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-17T00:00:09.502 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000301064 s, 1.7 MB/s 2023-11-17T00:00:09.504 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-17T00:00:09.558 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-16 23:59:12.751043104 +0000 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-16 23:53:21.787189412 +0000 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-16 23:53:21.787189412 +0000 2023-11-17T00:00:09.603 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-16 23:53:21.760189694 +0000 2023-11-17T00:00:09.603 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-17T00:00:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-17T00:00:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-17T00:00:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.00032044 s, 1.6 MB/s 2023-11-17T00:00:09.715 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-17T00:00:09.762 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2023-11-17T00:00:09.762 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2023-11-17T00:00:09.789 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-17T00:00:09.789 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 386477 Links: 1 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-11-16 23:59:54.721593693 +0000 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-11-16 23:49:52.406963576 +0000 2023-11-17T00:00:09.850 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-11-16 23:49:52.406963576 +0000 2023-11-17T00:00:09.851 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-11-17T00:00:09.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-17T00:00:09.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-11-17T00:00:09.921 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-11-17T00:00:09.921 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000245276 s, 2.1 MB/s 2023-11-17T00:00:09.922 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-17T00:00:09.986 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 393365 Links: 1 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-11-16 23:59:54.721593693 +0000 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-11-16 23:49:52.956950483 +0000 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-11-16 23:49:52.956950483 +0000 2023-11-17T00:00:10.046 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-11-17T00:00:10.047 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-17T00:00:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-11-17T00:00:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-11-17T00:00:10.115 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00023696 s, 2.2 MB/s 2023-11-17T00:00:10.116 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-17T00:00:10.175 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2023-11-17T00:00:10.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi087=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:10.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: Saving service mgr spec with placement smithi006=x;smithi087=y;smithi186=z;count:3 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 391077 Links: 1 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-11-16 23:59:54.722593669 +0000 2023-11-17T00:00:10.235 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-11-16 23:49:53.469938271 +0000 2023-11-17T00:00:10.236 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-11-16 23:49:53.469938271 +0000 2023-11-17T00:00:10.236 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-11-17T00:00:10.236 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi087=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: Saving service mgr spec with placement smithi006=x;smithi087=y;smithi186=z;count:3 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:10.301 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-11-17T00:00:10.303 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-11-17T00:00:10.303 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000231347 s, 2.2 MB/s 2023-11-17T00:00:10.304 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-17T00:00:10.363 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2023-11-17T00:00:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi006=x;smithi087=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: Saving service mgr spec with placement smithi006=x;smithi087=y;smithi186=z;count:3 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:10.420 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 394349 Links: 1 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2023-11-16 23:59:54.722593669 +0000 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2023-11-16 23:49:53.989925892 +0000 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2023-11-16 23:49:53.989925892 +0000 2023-11-17T00:00:10.421 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2023-11-17T00:00:10.421 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-17T00:00:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2023-11-17T00:00:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2023-11-17T00:00:10.488 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000187532 s, 2.7 MB/s 2023-11-17T00:00:10.488 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-17T00:00:10.545 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-11-17T00:00:10.546 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-11-17T00:00:10.562 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-17T00:00:10.563 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-11-17T00:00:10.620 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-17T00:00:10.620 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.620 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 379831 Links: 1 2023-11-17T00:00:10.620 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.620 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.621 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-16 23:59:46.566151478 +0000 2023-11-17T00:00:10.621 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-16 23:49:52.393960351 +0000 2023-11-17T00:00:10.621 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-16 23:49:52.393960351 +0000 2023-11-17T00:00:10.621 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-17T00:00:10.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-17T00:00:10.688 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-17T00:00:10.688 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-17T00:00:10.688 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000239725 s, 2.1 MB/s 2023-11-17T00:00:10.689 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-17T00:00:10.747 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-11-17T00:00:10.805 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-17T00:00:10.805 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.805 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 385763 Links: 1 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-16 23:59:46.567151457 +0000 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-16 23:49:52.926949145 +0000 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-16 23:49:52.926949145 +0000 2023-11-17T00:00:10.806 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-17T00:00:10.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-17T00:00:10.871 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-17T00:00:10.871 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-17T00:00:10.872 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000219146 s, 2.3 MB/s 2023-11-17T00:00:10.873 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-17T00:00:10.933 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 385900 Links: 1 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-16 23:59:46.567151457 +0000 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-16 23:49:53.440938338 +0000 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-16 23:49:53.440938338 +0000 2023-11-17T00:00:10.990 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-17T00:00:10.991 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-17T00:00:11.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:10 smithi186 ceph-mon[131487]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:11.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:10 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:11.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:10 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:00:11.044 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:10 smithi186 ceph-mon[131487]: Deploying daemon mgr.z on smithi186 2023-11-17T00:00:11.059 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-17T00:00:11.059 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-17T00:00:11.060 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000247826 s, 2.1 MB/s 2023-11-17T00:00:11.060 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-17T00:00:11.120 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 394393 Links: 1 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-11-17 00:00:11.107638701 +0000 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-11-16 23:49:53.971927174 +0000 2023-11-17T00:00:11.177 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-11-16 23:49:53.971927174 +0000 2023-11-17T00:00:11.178 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-11-17T00:00:11.178 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-17T00:00:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:10 smithi006 ceph-mon[97840]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:10 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:10 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:00:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:10 smithi006 ceph-mon[97840]: Deploying daemon mgr.z on smithi186 2023-11-17T00:00:11.295 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-11-17T00:00:11.295 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-11-17T00:00:11.295 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00018968 s, 2.7 MB/s 2023-11-17T00:00:11.296 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-17T00:00:11.314 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2023-11-17T00:00:11.314 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-17T00:00:11.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:10 smithi087 ceph-mon[131182]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:10 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:10 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:00:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:10 smithi087 ceph-mon[131182]: Deploying daemon mgr.z on smithi186 2023-11-17T00:00:11.557 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 systemd[1]: Starting Ceph mgr.z for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:00:11.847 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 podman[133619]: 2023-11-17T00:00:12.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 bash[133619]: 1156cd27c013a7838f2500223c94970e6f68d96150cdf2b3afb810d283234841 2023-11-17T00:00:12.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 ceph-mgr[133659]: -- 172.21.15.186:0/1677716762 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562694a11520 con 0x5626949e7000 2023-11-17T00:00:12.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 systemd[1]: Started Ceph mgr.z for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:00:12.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:11 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:11.971+0000 7f7a8c5ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-17T00:00:12.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.051+0000 7f7a8c5ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-17T00:00:12.175 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:00:12.401 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.140+0000 7f7a8c5ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-17T00:00:12.401 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.231+0000 7f7a8c5ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-17T00:00:12.708 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.400+0000 7f7a8c5ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-17T00:00:12.708 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.707+0000 7f7a8c5ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-17T00:00:12.928 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:12.928 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:12.929 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:13.022 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.795+0000 7f7a8c5ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-17T00:00:13.022 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:12.927+0000 7f7a8c5ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:13.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:13.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.241 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:13.242 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:13.242 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-17T00:00:13.242 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:13.242 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:13.365 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:13.021+0000 7f7a8c5ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-17T00:00:13.601 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:13 smithi087 systemd[1]: Starting Ceph mgr.y for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:00:13.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:13.438+0000 7f7a8c5ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-17T00:00:14.038 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:13 smithi087 ceph-mon[131182]: Deploying daemon mgr.y on smithi087 2023-11-17T00:00:14.038 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:13 smithi087 podman[132625]: 2023-11-17T00:00:14.038 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:13 smithi087 bash[132625]: 850a924c938f4c234376610bc34c200911f036dc134fb069a33e8813ac4ed550 2023-11-17T00:00:14.038 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:13 smithi087 ceph-mgr[132666]: -- 172.21.15.87:0/140853931 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc46195520 con 0x55bc4616b000 2023-11-17T00:00:14.187 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:13 smithi186 ceph-mon[131487]: Deploying daemon mgr.y on smithi087 2023-11-17T00:00:14.187 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:14.014+0000 7f7a8c5ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-17T00:00:14.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:13 smithi006 ceph-mon[97840]: Deploying daemon mgr.y on smithi087 2023-11-17T00:00:14.351 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 systemd[1]: Started Ceph mgr.y for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:00:14.351 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.095+0000 7f924339a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-17T00:00:14.351 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.174+0000 7f924339a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-17T00:00:14.351 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.264+0000 7f924339a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-17T00:00:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-17T00:00:14.565 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-17T00:00:14.566 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2023-11-17T00:00:14.566 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2023-11-17T00:00:14.566 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0306409 s, 342 MB/s 2023-11-17T00:00:14.566 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2023-11-17T00:00:14.615 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:14.185+0000 7f7a8c5ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-17T00:00:14.850 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.356+0000 7f924339a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-17T00:00:14.850 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.535+0000 7f924339a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-17T00:00:15.167 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2023-11-17T00:00:15.187 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.862+0000 7f924339a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-17T00:00:15.187 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:14.956+0000 7f924339a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-17T00:00:15.187 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:15.088+0000 7f924339a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.188 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:15.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:15.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:15.271 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:14.989+0000 7f7a8c5ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:00:15.271 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:15.092+0000 7f7a8c5ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-17T00:00:15.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:15.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.271 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:15.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:15.562 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:15.269+0000 7f7a8c5ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:00:15.562 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:15.560+0000 7f7a8c5ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-17T00:00:15.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:15.181+0000 7f924339a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-17T00:00:15.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:15.797+0000 7f7a8c5ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-17T00:00:15.923 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:15.635+0000 7f924339a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-17T00:00:16.026 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:00:16.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:16.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.221 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.221 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:15.955+0000 7f7a8c5ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-17T00:00:16.222 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:16.219+0000 7f7a8c5ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-17T00:00:16.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:15 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:16.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.223 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:16.223 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.223 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:16.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:16.221+0000 7f924339a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-17T00:00:16.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:16.392+0000 7f924339a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-17T00:00:17.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:17.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: Reconfiguring mgr.x (unknown last config time)... 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: Reconfiguring daemon mgr.x on smithi006 2023-11-17T00:00:17.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.238 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:17.145+0000 7f924339a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: Reconfiguring mgr.x (unknown last config time)... 2023-11-17T00:00:17.240 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:17.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:17.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: Reconfiguring daemon mgr.x on smithi006 2023-11-17T00:00:17.241 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.247 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:16.927+0000 7f7a8c5ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-17T00:00:17.247 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:17.056+0000 7f7a8c5ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-17T00:00:17.247 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:17.246+0000 7f7a8c5ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-17T00:00:17.247 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:17.247 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:17.248 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: Reconfiguring mgr.x (unknown last config time)... 2023-11-17T00:00:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-17T00:00:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-17T00:00:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: Reconfiguring daemon mgr.x on smithi006 2023-11-17T00:00:17.249 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:17.575 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:17.359+0000 7f7a8c5ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-17T00:00:17.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:17.234+0000 7f924339a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-17T00:00:17.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:17.401+0000 7f924339a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:00:17.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:17.574+0000 7f7a8c5ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-17T00:00:18.041 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:17.674+0000 7f924339a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-17T00:00:18.041 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:17.896+0000 7f924339a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-17T00:00:18.299 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:18.041+0000 7f924339a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 -- 172.21.15.6:0/1803738337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1014f0 msgr2=0x7ffb4c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1803738337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c101940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009990 tx=0x7ffb40009ca0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 -- 172.21.15.6:0/1803738337 shutdown_connections 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1803738337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c101e80 0x7ffb4c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1803738337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1803738337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c101940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 -- 172.21.15.6:0/1803738337 >> 172.21.15.6:0/1803738337 conn(0x7ffb4c0fb900 msgr2=0x7ffb4c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 -- 172.21.15.6:0/1803738337 shutdown_connections 2023-11-17T00:00:18.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.355+0000 7ffb50f14700 1 -- 172.21.15.6:0/1803738337 wait complete. 2023-11-17T00:00:18.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.356+0000 7ffb50f14700 1 Processor -- start 2023-11-17T00:00:18.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 -- start start 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c101e80 0x7ffb4c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c073ba0 con 0x7ffb4c1002f0 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c073d10 con 0x7ffb4c1014f0 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb50f14700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1a00e0 con 0x7ffb4c101e80 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb4a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:18.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb49d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb4ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c101e80 0x7ffb4c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb4a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56236/0 (socket says 172.21.15.6:56236) 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb49d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:54574/0 (socket says 172.21.15.6:54574) 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.357+0000 7ffb4a59c700 1 -- 172.21.15.6:0/1516003012 learned_addr learned my addr 172.21.15.6:0/1516003012 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 -- 172.21.15.6:0/1516003012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c101e80 msgr2=0x7ffb4c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c101e80 0x7ffb4c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 -- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 msgr2=0x7ffb4c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 -- 172.21.15.6:0/1516003012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400095f0 con 0x7ffb4c1014f0 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb49d9b700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000b680 tx=0x7ffb40009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4001d070 con 0x7ffb4c1014f0 2023-11-17T00:00:18.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb40021410 con 0x7ffb4c1014f0 2023-11-17T00:00:18.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40017650 con 0x7ffb4c1014f0 2023-11-17T00:00:18.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c10bc90 con 0x7ffb4c1014f0 2023-11-17T00:00:18.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.358+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c10bb00 con 0x7ffb4c1014f0 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.359+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7ffb40021810 con 0x7ffb4c1014f0 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.359+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c1a0620 con 0x7ffb4c1014f0 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.360+0000 7ffb3b7fe700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 0x7ffb3403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.360+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb4004c6d0 con 0x7ffb4c1014f0 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.360+0000 7ffb4a59c700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 0x7ffb3403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:18.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.360+0000 7ffb4a59c700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 0x7ffb3403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005950 tx=0x7ffb3c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.365+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffb40029540 con 0x7ffb4c1014f0 2023-11-17T00:00:18.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.591+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb4c060f30 con 0x7ffb340389e0 2023-11-17T00:00:18.600 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:18.299+0000 7f924339a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-17T00:00:18.861 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:00:18.440+0000 7f7a8c5ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-17T00:00:18.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:18.934+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 14) v1 ==== 90598+0+0 (secure 0 0 0) 0x7ffb4002b440 con 0x7ffb4c1014f0 2023-11-17T00:00:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: Standby manager daemon z started 2023-11-17T00:00:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:00:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:18.998+0000 7f924339a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: Standby manager daemon z started 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.127 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:19.128 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:19.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: Standby manager daemon z started 2023-11-17T00:00:19.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/2081740472' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:19.425 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:19.126+0000 7f924339a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-17T00:00:19.426 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:19.314+0000 7f924339a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-17T00:00:19.851 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:19.425+0000 7f924339a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-17T00:00:19.851 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:19.637+0000 7f924339a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-17T00:00:20.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:00:20.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:20.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: mgrmap e14: x(active, since 77s), standbys: z 2023-11-17T00:00:20.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:19 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:00:20.182 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.182 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:00:20.183 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.183 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:20.183 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: mgrmap e14: x(active, since 77s), standbys: z 2023-11-17T00:00:20.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:19 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:00:20.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:00:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: mgrmap e14: x(active, since 77s), standbys: z 2023-11-17T00:00:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:19 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:00:20.850 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:00:20.477+0000 7f924339a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-17T00:00:20.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:20.941+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mon.2 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7ffb4000fad0 con 0x7ffb4c1014f0 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: Standby manager daemon y started 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:00:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:20 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: Standby manager daemon y started 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:00:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:20 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: Standby manager daemon y started 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:00:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:20 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/1656869998' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:00:22.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:21 smithi006 ceph-mon[97840]: mgrmap e15: x(active, since 79s), standbys: z, y 2023-11-17T00:00:22.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:00:22.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:21 smithi006 ceph-mon[97840]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:22.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:21 smithi087 ceph-mon[131182]: mgrmap e15: x(active, since 79s), standbys: z, y 2023-11-17T00:00:22.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:00:22.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:21 smithi087 ceph-mon[131182]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:22.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:21 smithi186 ceph-mon[131487]: mgrmap e15: x(active, since 79s), standbys: z, y 2023-11-17T00:00:22.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:00:22.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:21 smithi186 ceph-mon[131487]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:23.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]: dispatch 2023-11-17T00:00:23.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]': finished 2023-11-17T00:00:23.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:22 smithi006 ceph-mon[97840]: osdmap e5: 1 total, 0 up, 1 in 2023-11-17T00:00:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3456803302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:22 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]: dispatch 2023-11-17T00:00:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:22 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]': finished 2023-11-17T00:00:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:22 smithi087 ceph-mon[131182]: osdmap e5: 1 total, 0 up, 1 in 2023-11-17T00:00:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:22 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3456803302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:22 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]: dispatch 2023-11-17T00:00:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:22 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/526122084' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2da7d9-d6af-4949-9a5f-d392c2dcc273"}]': finished 2023-11-17T00:00:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:22 smithi186 ceph-mon[131487]: osdmap e5: 1 total, 0 up, 1 in 2023-11-17T00:00:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:22 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3456803302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:23 smithi006 ceph-mon[97840]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:23 smithi087 ceph-mon[131182]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:24.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:23 smithi186 ceph-mon[131487]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:26.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:26 smithi006 ceph-mon[97840]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:27.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:26 smithi087 ceph-mon[131182]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:26 smithi186 ceph-mon[131487]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:28 smithi186 ceph-mon[131487]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:29.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:28 smithi006 ceph-mon[97840]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:29.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:28 smithi087 ceph-mon[131182]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:31.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:30 smithi006 ceph-mon[97840]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:31.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-17T00:00:31.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:31.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:30 smithi087 ceph-mon[131182]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:31.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-17T00:00:31.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:31.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:30 smithi186 ceph-mon[131487]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:31.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-17T00:00:31.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:32.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:31 smithi006 ceph-mon[97840]: Deploying daemon osd.0 on smithi006 2023-11-17T00:00:32.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:31 smithi087 ceph-mon[131182]: Deploying daemon osd.0 on smithi006 2023-11-17T00:00:32.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:31 smithi186 ceph-mon[131487]: Deploying daemon osd.0 on smithi006 2023-11-17T00:00:33.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:32 smithi006 ceph-mon[97840]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:33.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:32 smithi087 ceph-mon[131182]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:32 smithi186 ceph-mon[131487]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:34 smithi006 ceph-mon[97840]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:34 smithi087 ceph-mon[131182]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:34 smithi186 ceph-mon[131487]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:36.970 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:36 smithi006 ceph-mon[97840]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:37.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:36 smithi087 ceph-mon[131182]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:37.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:36 smithi186 ceph-mon[131487]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:38.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:38.431 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:38 smithi006 ceph-mon[97840]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:38 smithi087 ceph-mon[131182]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:38.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:38.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:38.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:38 smithi186 ceph-mon[131487]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:41.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.059+0000 7ffb3b7fe700 1 -- 172.21.15.6:0/1516003012 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb4c060f30 con 0x7ffb340389e0 2023-11-17T00:00:41.060 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2023-11-17T00:00:41.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 msgr2=0x7ffb3403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 0x7ffb3403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c005950 tx=0x7ffb3c009260 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 msgr2=0x7ffb4c072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000b680 tx=0x7ffb40009580 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 shutdown_connections 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffb340389e0 0x7ffb3403ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb4c101e80 0x7ffb4c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 --2- 172.21.15.6:0/1516003012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c072ee0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 >> 172.21.15.6:0/1516003012 conn(0x7ffb4c0fb900 msgr2=0x7ffb4c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 shutdown_connections 2023-11-17T00:00:41.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:00:41.062+0000 7ffb50f14700 1 -- 172.21.15.6:0/1516003012 wait complete. 2023-11-17T00:00:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:00:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:40 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:00:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:40 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:00:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:40 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:41.640 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.0.service 2023-11-17T00:00:41.645 INFO:tasks.cephadm:Deploying osd.1 on smithi087 with /dev/vg_nvme/lv_4... 2023-11-17T00:00:41.645 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-17T00:00:41.722 INFO:journalctl@ceph.osd.0.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:00:42.289 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:00:42.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.326 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:42.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:42 smithi006 ceph-mon[97840]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-17T00:00:42.327 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:42 smithi006 ceph-mon[97840]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:42.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:42 smithi087 ceph-mon[131182]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-17T00:00:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:42 smithi087 ceph-mon[131182]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:00:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:42 smithi186 ceph-mon[131487]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-17T00:00:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:42 smithi186 ceph-mon[131487]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:43.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:43 smithi087 ceph-mon[131182]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-17T00:00:43.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:43 smithi087 ceph-mon[131182]: osdmap e6: 1 total, 0 up, 1 in 2023-11-17T00:00:43.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:43.344 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:43 smithi087 ceph-mon[131182]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:00:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:43 smithi186 ceph-mon[131487]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-17T00:00:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:43 smithi186 ceph-mon[131487]: osdmap e6: 1 total, 0 up, 1 in 2023-11-17T00:00:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:43 smithi186 ceph-mon[131487]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:00:43.474 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0[110097]: 2023-11-17T00:00:43.060+0000 7fa1069b7700 -1 osd.0 0 waiting for initial osdmap 2023-11-17T00:00:43.474 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0[110097]: 2023-11-17T00:00:43.065+0000 7fa1007a8700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:00:43.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-mon[97840]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-17T00:00:43.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-mon[97840]: osdmap e6: 1 total, 0 up, 1 in 2023-11-17T00:00:43.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:43.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:43 smithi006 ceph-mon[97840]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112325 s, 934 MB/s 2023-11-17T00:00:43.756 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-11-17T00:00:44.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_4 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: osdmap e7: 1 total, 0 up, 1 in 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:44.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:00:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: osdmap e7: 1 total, 0 up, 1 in 2023-11-17T00:00:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: from='osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: osdmap e7: 1 total, 0 up, 1 in 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-17T00:00:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:44.773 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:00:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:00:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:00:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] boot 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: osdmap e8: 1 total, 1 up, 1 in 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] boot 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: osdmap e8: 1 total, 1 up, 1 in 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:00:45.270 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.271 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:45.271 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:45.271 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: osd.0 [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] boot 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: osdmap e8: 1 total, 1 up, 1 in 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:00:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:00:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:00:45.912 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 -- 172.21.15.87:0/2231133229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 msgr2=0x7fcc88105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:45.912 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 --2- 172.21.15.87:0/2231133229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc88105d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009990 tx=0x7fcc84009ca0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.912 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 -- 172.21.15.87:0/2231133229 shutdown_connections 2023-11-17T00:00:45.912 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 --2- 172.21.15.87:0/2231133229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc88106250 0x7fcc88108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 --2- 172.21.15.87:0/2231133229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 0x7fcc881033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 --2- 172.21.15.87:0/2231133229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc88105d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 -- 172.21.15.87:0/2231133229 >> 172.21.15.87:0/2231133229 conn(0x7fcc880faa50 msgr2=0x7fcc880fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 -- 172.21.15.87:0/2231133229 shutdown_connections 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.909+0000 7fcc906b1700 1 -- 172.21.15.87:0/2231133229 wait complete. 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 Processor -- start 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 -- start start 2023-11-17T00:00:45.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 0x7fcc88196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc88106250 0x7fcc8819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8819de00 con 0x7fcc88106250 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8819df40 con 0x7fcc88103930 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc906b1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8819e0d0 con 0x7fcc88101010 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35384/0 (socket says 172.21.15.87:35384) 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 -- 172.21.15.87:0/100784218 learned_addr learned my addr 172.21.15.87:0/100784218 (peer_addr_for_me v2:172.21.15.87:0/0) 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8e44d700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 0x7fcc88196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:45.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 -- 172.21.15.87:0/100784218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 msgr2=0x7fcc88196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 0x7fcc88196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 -- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc88106250 msgr2=0x7fcc8819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc88106250 0x7fcc8819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.910+0000 7fcc8dc4c700 1 -- 172.21.15.87:0/100784218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840095f0 con 0x7fcc88103930 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc8dc4c700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000c00 tx=0x7fcc8400f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401d070 con 0x7fcc88103930 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8400fc00 con 0x7fcc88103930 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8810bf10 con 0x7fcc88103930 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84017650 con 0x7fcc88103930 2023-11-17T00:00:45.915 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.911+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8810bd80 con 0x7fcc88103930 2023-11-17T00:00:45.916 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.912+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880fc5c0 con 0x7fcc88103930 2023-11-17T00:00:45.916 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.913+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7fcc84022470 con 0x7fcc88103930 2023-11-17T00:00:45.916 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.913+0000 7fcc7f7fe700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 0x7fcc740a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:00:45.916 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.913+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fcc840cc5c0 con 0x7fcc88103930 2023-11-17T00:00:45.917 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.914+0000 7fcc8e44d700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 0x7fcc740a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:00:45.917 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.914+0000 7fcc8e44d700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 0x7fcc740a3570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc78005950 tx=0x7fcc780058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:00:45.918 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:45.917+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcc84097810 con 0x7fcc88103930 2023-11-17T00:00:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:00:46.110+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc88060f30 con 0x7fcc740a10c0 2023-11-17T00:00:46.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi006 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 17276M 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: osdmap e9: 1 total, 1 up, 1 in 2023-11-17T00:00:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:46 smithi087 ceph-mon[131182]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi006 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 17276M 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: osdmap e9: 1 total, 1 up, 1 in 2023-11-17T00:00:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:46 smithi186 ceph-mon[131487]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:46.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi006 2023-11-17T00:00:46.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 17276M 2023-11-17T00:00:46.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:00:46.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:00:46.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: osdmap e9: 1 total, 1 up, 1 in 2023-11-17T00:00:46.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:46 smithi006 ceph-mon[97840]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:47.248 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:47 smithi087 ceph-mon[131182]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:47.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:47.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:47.249 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:47 smithi186 ceph-mon[131487]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:47.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:47 smithi006 ceph-mon[97840]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:00:47.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:00:47.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:00:47.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:49.228 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:49.230 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/3006095726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.231 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.231 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]': finished 2023-11-17T00:00:49.231 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: osdmap e10: 2 total, 1 up, 2 in 2023-11-17T00:00:49.232 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:48 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:00:49.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/3006095726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]': finished 2023-11-17T00:00:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: osdmap e10: 2 total, 1 up, 2 in 2023-11-17T00:00:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:48 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/3006095726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]: dispatch 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbdc1ecd-4466-47fc-8510-60d676144e56"}]': finished 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: osdmap e10: 2 total, 1 up, 2 in 2023-11-17T00:00:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:48 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:00:50.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:49 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/3138708311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:50.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:49 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/3138708311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:50.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:49 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/3138708311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:00:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:50 smithi006 ceph-mon[97840]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:51.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:50 smithi087 ceph-mon[131182]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:51.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:50 smithi186 ceph-mon[131487]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:52 smithi006 ceph-mon[97840]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:53.318 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:52 smithi087 ceph-mon[131182]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:53.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:52 smithi186 ceph-mon[131487]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:54 smithi087 ceph-mon[131182]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:55.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:54 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-17T00:00:55.253 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:54 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:55.254 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:54 smithi006 ceph-mon[97840]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:55.254 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:54 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-17T00:00:55.254 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:54 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:55.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:54 smithi186 ceph-mon[131487]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:55.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:54 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-17T00:00:55.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:54 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:00:56.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:55 smithi087 ceph-mon[131182]: Deploying daemon osd.1 on smithi087 2023-11-17T00:00:56.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:55 smithi006 ceph-mon[97840]: Deploying daemon osd.1 on smithi087 2023-11-17T00:00:56.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:55 smithi186 ceph-mon[131487]: Deploying daemon osd.1 on smithi087 2023-11-17T00:00:56.993 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:56 smithi087 ceph-mon[131182]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:57.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:56 smithi006 ceph-mon[97840]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:57.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:56 smithi186 ceph-mon[131487]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:59.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:00:58 smithi087 ceph-mon[131182]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:59.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:00:58 smithi006 ceph-mon[97840]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:00:59.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:00:58 smithi186 ceph-mon[131487]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:00.257 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.257 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.257 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:00.258 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:00 smithi087 ceph-mon[131182]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:00.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:00.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:00 smithi186 ceph-mon[131487]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:00.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:00.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:00 smithi006 ceph-mon[97840]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:01.825 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 1 on host 'smithi087' 2023-11-17T00:01:01.825 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.824+0000 7fcc7f7fe700 1 -- 172.21.15.87:0/100784218 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcc88060f30 con 0x7fcc740a10c0 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 msgr2=0x7fcc740a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 0x7fcc740a3570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc78005950 tx=0x7fcc780058e0 comp rx=0 tx=0).stop 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 msgr2=0x7fcc881967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000c00 tx=0x7fcc8400f710 comp rx=0 tx=0).stop 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 shutdown_connections 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcc740a10c0 0x7fcc740a3570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:01.827 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcc88106250 0x7fcc8819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc88101010 0x7fcc88196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 --2- 172.21.15.87:0/100784218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc88103930 0x7fcc881967b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 >> 172.21.15.87:0/100784218 conn(0x7fcc880faa50 msgr2=0x7fcc88101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:01:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 shutdown_connections 2023-11-17T00:01:01.828 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:01:01.827+0000 7fcc906b1700 1 -- 172.21.15.87:0/100784218 wait complete. 2023-11-17T00:01:02.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.224 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.225 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:02.226 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:02.226 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.226 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.227 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:01:02.227 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.227 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:02.227 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.228 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.228 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:02.247 DEBUG:teuthology.orchestra.run.smithi087:osd.1> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.1.service 2023-11-17T00:01:02.250 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2023-11-17T00:01:02.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-17T00:01:02.285 INFO:journalctl@ceph.osd.1.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:01:02.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:02.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:02.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:02.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:02.905 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:01:03.350 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:03 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:01:03.054+0000 7fdf15e5a640 -1 osd.1 0 log_to_monitors true 2023-11-17T00:01:04.155 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:03 smithi087 ceph-mon[131182]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.155 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:03 smithi087 ceph-mon[131182]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:03 smithi006 ceph-mon[97840]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:03 smithi006 ceph-mon[97840]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.341 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-17T00:01:04.341 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-17T00:01:04.341 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-11-17T00:01:04.341 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-11-17T00:01:04.342 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127495 s, 822 MB/s 2023-11-17T00:01:04.342 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-11-17T00:01:04.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:03 smithi186 ceph-mon[131487]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:03 smithi186 ceph-mon[131487]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-17T00:01:04.760 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2023-11-17T00:01:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: pgmap v61: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: osdmap e11: 2 total, 1 up, 2 in 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:05.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: pgmap v61: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: osdmap e11: 2 total, 1 up, 2 in 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.351 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:01:04.886+0000 7fdf0b4d1700 -1 osd.1 0 waiting for initial osdmap 2023-11-17T00:01:05.351 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:01:04.890+0000 7fdf032be700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: pgmap v61: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: osdmap e11: 2 total, 1 up, 2 in 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:01:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:05.402 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:01:06.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi087 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 17168M 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: osdmap e12: 2 total, 1 up, 2 in 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi087 2023-11-17T00:01:06.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 17168M 2023-11-17T00:01:06.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:01:06.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:01:06.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: osdmap e12: 2 total, 1 up, 2 in 2023-11-17T00:01:06.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi087 2023-11-17T00:01:06.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 17168M 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: osdmap e12: 2 total, 1 up, 2 in 2023-11-17T00:01:06.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 -- 172.21.15.186:0/4056569791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 msgr2=0x7f882c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 --2- 172.21.15.186:0/4056569791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c100720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f881c009990 tx=0x7f881c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 -- 172.21.15.186:0/4056569791 shutdown_connections 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 --2- 172.21.15.186:0/4056569791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 --2- 172.21.15.186:0/4056569791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c100720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 --2- 172.21.15.186:0/4056569791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 0x7f882c101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 -- 172.21.15.186:0/4056569791 >> 172.21.15.186:0/4056569791 conn(0x7f882c0fb960 msgr2=0x7f882c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 -- 172.21.15.186:0/4056569791 shutdown_connections 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.633+0000 7f88344b3700 1 -- 172.21.15.186:0/4056569791 wait complete. 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.634+0000 7f88344b3700 1 Processor -- start 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.634+0000 7f88344b3700 1 -- start start 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.634+0000 7f88344b3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.634+0000 7f88344b3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 0x7f882c196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88344b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:06.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88344b3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19dc70 con 0x7f882c101ea0 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88344b3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19ddb0 con 0x7f882c101510 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88344b3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19df40 con 0x7f882c100310 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55600/0 (socket says 172.21.15.186:55600) 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 -- 172.21.15.186:0/4059497922 learned_addr learned my addr 172.21.15.186:0/4059497922 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 -- 172.21.15.186:0/4059497922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 msgr2=0x7f882c196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f8831a4e700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 0x7f882c196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:06.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 0x7f882c196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f8832a50700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 -- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 msgr2=0x7f882c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 -- 172.21.15.186:0/4059497922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0095f0 con 0x7f882c100310 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f8832a50700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:01:06.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f883224f700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f881c00b5c0 tx=0x7f881c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c01d070 con 0x7f882c100310 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881c022470 con 0x7f882c100310 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c10bce0 con 0x7f882c100310 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.635+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c017630 con 0x7f882c100310 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.636+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c10bb50 con 0x7f882c100310 2023-11-17T00:01:06.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.637+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c066ea0 con 0x7f882c100310 2023-11-17T00:01:06.644 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.637+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f881c0228e0 con 0x7f882c100310 2023-11-17T00:01:06.645 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.638+0000 7f88237fe700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 0x7f88180a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:06.645 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.638+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f881c0cc4b0 con 0x7f882c100310 2023-11-17T00:01:06.645 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.638+0000 7f8831a4e700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 0x7f88180a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:06.645 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.639+0000 7f8831a4e700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 0x7f88180a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8828005950 tx=0x7f88280058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:01:06.646 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.642+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f881c097600 con 0x7f882c100310 2023-11-17T00:01:06.837 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:06.835+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f882c060f30 con 0x7f88180a10c0 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: pgmap v64: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] boot 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: osdmap e13: 2 total, 2 up, 2 in 2023-11-17T00:01:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:07.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:07.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:07.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: pgmap v64: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] boot 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: osdmap e13: 2 total, 2 up, 2 in 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:01:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:01:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: pgmap v64: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-17T00:01:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: osd.1 [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] boot 2023-11-17T00:01:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: osdmap e13: 2 total, 2 up, 2 in 2023-11-17T00:01:07.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:01:07.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:07.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:07.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:07 smithi186 ceph-mon[131487]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:07.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:07 smithi186 ceph-mon[131487]: osdmap e14: 2 total, 2 up, 2 in 2023-11-17T00:01:08.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:07 smithi006 ceph-mon[97840]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:07 smithi006 ceph-mon[97840]: osdmap e14: 2 total, 2 up, 2 in 2023-11-17T00:01:08.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:07 smithi087 ceph-mon[131182]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:07 smithi087 ceph-mon[131182]: osdmap e14: 2 total, 2 up, 2 in 2023-11-17T00:01:09.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:08 smithi186 ceph-mon[131487]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:08 smithi006 ceph-mon[97840]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:09.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:08 smithi087 ceph-mon[131182]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:10.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/3287192825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:09 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:09 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]': finished 2023-11-17T00:01:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:09 smithi006 ceph-mon[97840]: osdmap e15: 3 total, 2 up, 3 in 2023-11-17T00:01:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:09 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/3287192825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:09 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:09 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]': finished 2023-11-17T00:01:10.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:09 smithi087 ceph-mon[131182]: osdmap e15: 3 total, 2 up, 3 in 2023-11-17T00:01:10.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:09 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/3287192825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:09 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]: dispatch 2023-11-17T00:01:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:09 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17362d0b-f379-4a0e-a815-aec6d56732e4"}]': finished 2023-11-17T00:01:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:09 smithi186 ceph-mon[131487]: osdmap e15: 3 total, 2 up, 3 in 2023-11-17T00:01:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:10 smithi006 ceph-mon[97840]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:10 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/1071008441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:11.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:10 smithi087 ceph-mon[131182]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:10 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/1071008441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:10 smithi186 ceph-mon[131487]: pgmap v69: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:10 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/1071008441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:13.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:12 smithi006 ceph-mon[97840]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:13.298 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:12 smithi186 ceph-mon[131487]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:12 smithi087 ceph-mon[131182]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:15.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:14 smithi186 ceph-mon[131487]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:15.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:14 smithi006 ceph-mon[97840]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:15.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:14 smithi087 ceph-mon[131182]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:17.064 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:16 smithi186 ceph-mon[131487]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:17.064 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-17T00:01:17.065 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:16 smithi006 ceph-mon[97840]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-17T00:01:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:17.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:16 smithi087 ceph-mon[131182]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-17T00:01:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:18.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:17 smithi006 ceph-mon[97840]: Deploying daemon osd.2 on smithi186 2023-11-17T00:01:18.320 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:17 smithi186 ceph-mon[131487]: Deploying daemon osd.2 on smithi186 2023-11-17T00:01:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:17 smithi087 ceph-mon[131182]: Deploying daemon osd.2 on smithi186 2023-11-17T00:01:19.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:18 smithi006 ceph-mon[97840]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:19.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:18 smithi087 ceph-mon[131182]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:19.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:18 smithi186 ceph-mon[131487]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:21.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:20 smithi006 ceph-mon[97840]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:21.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:20 smithi087 ceph-mon[131182]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:21.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:20 smithi186 ceph-mon[131487]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:21.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:21.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:23.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.313 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:23.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:23.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:23.314 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:23.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:23.602 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.601+0000 7f88237fe700 1 -- 172.21.15.186:0/4059497922 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f882c060f30 con 0x7f88180a10c0 2023-11-17T00:01:23.602 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 msgr2=0x7f88180a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 0x7f88180a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8828005950 tx=0x7f88280058e0 comp rx=0 tx=0).stop 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 msgr2=0x7f882c1960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f881c00b5c0 tx=0x7f881c00f690 comp rx=0 tx=0).stop 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 shutdown_connections 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f88180a10c0 0x7f88180a3570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f882c101ea0 0x7f882c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f882c100310 0x7f882c1960e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:23.604 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.603+0000 7f88344b3700 1 --2- 172.21.15.186:0/4059497922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f882c101510 0x7f882c196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:23.605 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.604+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 >> 172.21.15.186:0/4059497922 conn(0x7f882c0fb960 msgr2=0x7f882c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:01:23.605 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.604+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 shutdown_connections 2023-11-17T00:01:23.605 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:01:23.604+0000 7f88344b3700 1 -- 172.21.15.186:0/4059497922 wait complete. 2023-11-17T00:01:24.093 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.2.service 2023-11-17T00:01:24.095 INFO:tasks.cephadm:Deploying osd.3 on smithi006 with /dev/vg_nvme/lv_3... 2023-11-17T00:01:24.095 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-17T00:01:24.125 INFO:journalctl@ceph.osd.2.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:01:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:24.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:01:24.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:24.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:24.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:24.727 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:01:24.442+0000 7ff66ed2f640 -1 osd.2 0 log_to_monitors true 2023-11-17T00:01:29.692 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:01:29.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:24 smithi186 ceph-mon[131487]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:24 smithi186 ceph-mon[131487]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.693 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:24 smithi186 ceph-mon[131487]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.693 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:01:25.950+0000 7ff6643a6700 -1 osd.2 0 waiting for initial osdmap 2023-11-17T00:01:29.768 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:01:25.954+0000 7ff66019b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:01:29.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:24 smithi006 ceph-mon[97840]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:24 smithi006 ceph-mon[97840]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:24 smithi006 ceph-mon[97840]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-17T00:01:29.843 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: osdmap e16: 3 total, 2 up, 3 in 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:25 smithi006 ceph-mon[97840]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: osdmap e17: 3 total, 2 up, 3 in 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi186 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:29.844 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 17168M 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:26 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:29.845 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:24 smithi087 ceph-mon[131182]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:24 smithi087 ceph-mon[131182]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:24 smithi087 ceph-mon[131182]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: osdmap e16: 3 total, 2 up, 3 in 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:25 smithi087 ceph-mon[131182]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:01:29.846 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: osdmap e17: 3 total, 2 up, 3 in 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi186 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 17168M 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.847 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:26 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] boot 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: osdmap e18: 3 total, 3 up, 3 in 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:29 smithi087 ceph-mon[131182]: pgmap v81: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:29 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:29 smithi087 ceph-mon[131182]: osdmap e19: 3 total, 3 up, 3 in 2023-11-17T00:01:29.848 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:29 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:29.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-17T00:01:29.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: osdmap e16: 3 total, 2 up, 3 in 2023-11-17T00:01:29.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: from='osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.849 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:25 smithi186 ceph-mon[131487]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: osdmap e17: 3 total, 2 up, 3 in 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi186 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:29.925 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:01:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 17168M 2023-11-17T00:01:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:30.426 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:30.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:30.495 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:30.578 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:26 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] boot 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: osdmap e18: 3 total, 3 up, 3 in 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:30.579 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:30.632 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:29 smithi186 ceph-mon[131487]: pgmap v81: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:30.632 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:29 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:30.632 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:29 smithi186 ceph-mon[131487]: osdmap e19: 3 total, 3 up, 3 in 2023-11-17T00:01:30.633 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:29 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0465462 s, 225 MB/s 2023-11-17T00:01:30.633 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2023-11-17T00:01:30.634 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2023-11-17T00:01:30.635 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:01:30.635 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:01:30.660 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: osd.2 [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] boot 2023-11-17T00:01:30.765 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: osdmap e18: 3 total, 3 up, 3 in 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:29 smithi006 ceph-mon[97840]: pgmap v81: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:29 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:29 smithi006 ceph-mon[97840]: osdmap e19: 3 total, 3 up, 3 in 2023-11-17T00:01:30.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:29 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:01:30.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:30.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:30 smithi087 ceph-mon[131182]: osdmap e20: 3 total, 3 up, 3 in 2023-11-17T00:01:30.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:30 smithi087 ceph-mon[131182]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:30.767 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:30.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:30 smithi186 ceph-mon[131487]: osdmap e20: 3 total, 3 up, 3 in 2023-11-17T00:01:30.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:30 smithi186 ceph-mon[131487]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:31.139 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:29 smithi006 sudo[112833]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:31.139 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:29 smithi006 sudo[112833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.139 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:29 smithi006 sudo[112833]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.139 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:29 smithi006 sudo[112833]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:31.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:01:31.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:30 smithi006 ceph-mon[97840]: osdmap e20: 3 total, 3 up, 3 in 2023-11-17T00:01:31.140 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:30 smithi006 ceph-mon[97840]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:31.140 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:30 smithi006 sudo[112836]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:31.140 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:30 smithi006 sudo[112836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.140 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:30 smithi006 sudo[112836]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.140 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:01:30 smithi006 sudo[112836]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:31.141 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:30 smithi087 sudo[138551]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:31.141 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:30 smithi087 sudo[138551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.141 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:30 smithi087 sudo[138551]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.141 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:30 smithi087 sudo[138551]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:31.288 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:31 smithi087 sudo[138554]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:31.288 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:31 smithi087 sudo[138554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.288 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:31 smithi087 sudo[138554]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.288 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:01:31 smithi087 sudo[138554]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:31.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:31 smithi006 ceph-mon[97840]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-17T00:01:31.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:31 smithi006 ceph-mon[97840]: osdmap e21: 3 total, 3 up, 3 in 2023-11-17T00:01:31.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:31 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:31 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.519 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139462]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:31.519 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.519 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139462]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.519 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139462]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:31.519 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:31 smithi186 ceph-mon[131487]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-17T00:01:31.520 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:31 smithi186 ceph-mon[131487]: osdmap e21: 3 total, 3 up, 3 in 2023-11-17T00:01:31.520 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:31 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.520 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:31 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:31 smithi087 ceph-mon[131182]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-17T00:01:31.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:31 smithi087 ceph-mon[131182]: osdmap e21: 3 total, 3 up, 3 in 2023-11-17T00:01:31.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:31 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:31 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:31.865 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139465]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:31.865 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139465]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:31.865 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139465]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:31.865 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:01:31 smithi186 sudo[139465]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:32.016 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:01:32.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112910]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:32.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:32.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112910]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:32.432 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112910]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:32.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112947]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:32.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112947]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:32.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112947]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:32.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 sudo[112947]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:32.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:32.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:32 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 sudo[139468]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 sudo[139468]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 sudo[139468]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:32.926 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:32 smithi186 sudo[139468]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:32 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 sudo[139471]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 sudo[139471]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 sudo[139471]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 sudo[139471]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:33.654 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138557]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-17T00:01:33.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:33.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138557]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:33.655 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138557]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 -- 172.21.15.6:0/1694712708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494074aa0 msgr2=0x7f9494072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1694712708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494074aa0 0x7f9494072f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f9484009ca0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 -- 172.21.15.6:0/1694712708 shutdown_connections 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1694712708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 0x7f949410a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1694712708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9494074aa0 0x7f9494072f00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1694712708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94940734d0 0x7f9494073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 -- 172.21.15.6:0/1694712708 >> 172.21.15.6:0/1694712708 conn(0x7f94940fb9a0 msgr2=0x7f94940fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.795+0000 7f9498cc0700 1 -- 172.21.15.6:0/1694712708 shutdown_connections 2023-11-17T00:01:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.796+0000 7f9498cc0700 1 -- 172.21.15.6:0/1694712708 wait complete. 2023-11-17T00:01:33.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 Processor -- start 2023-11-17T00:01:33.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 -- start start 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940734d0 0x7f94941981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 0x7f949419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949419fd40 con 0x7f9494074aa0 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949419fe80 con 0x7f9494108230 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9498cc0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94941a0010 con 0x7f94940734d0 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9491d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9491d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:50440/0 (socket says 172.21.15.6:50440) 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.797+0000 7f9491d9b700 1 -- 172.21.15.6:0/1051281938 learned_addr learned my addr 172.21.15.6:0/1051281938 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9492d9d700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 0x7f949419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:33.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f949259c700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940734d0 0x7f94941981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 -- 172.21.15.6:0/1051281938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940734d0 msgr2=0x7f94941981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940734d0 0x7f94941981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 -- 172.21.15.6:0/1051281938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 msgr2=0x7f949419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 0x7f949419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 -- 172.21.15.6:0/1051281938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9488009520 con 0x7f9494074aa0 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9491d9b700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948800eb60 tx=0x7f948800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:01:33.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800cbd0 con 0x7f9494074aa0 2023-11-17T00:01:33.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94840095f0 con 0x7f9494074aa0 2023-11-17T00:01:33.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.798+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94880044e0 con 0x7f9494074aa0 2023-11-17T00:01:33.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.799+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488003d90 con 0x7f9494074aa0 2023-11-17T00:01:33.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.799+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949410deb0 con 0x7f9494074aa0 2023-11-17T00:01:33.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.801+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494106970 con 0x7f9494074aa0 2023-11-17T00:01:33.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.802+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9488003ef0 con 0x7f9494074aa0 2023-11-17T00:01:33.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.803+0000 7f94837fe700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 0x7f947c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:01:33.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.804+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f9488014070 con 0x7f9494074aa0 2023-11-17T00:01:33.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.804+0000 7f949259c700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 0x7f947c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:01:33.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.804+0000 7f949259c700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 0x7f947c0a3600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f948400b2e0 tx=0x7f9484005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:01:33.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:33.808+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f94880963f0 con 0x7f9494074aa0 2023-11-17T00:01:33.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:33.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:33.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:33 smithi006 ceph-mon[97840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:34.023+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9494060f30 con 0x7f947c0a1150 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138560]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138560]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 sudo[138560]: pam_unix(sudo:session): session closed for user root 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:33 smithi087 ceph-mon[131182]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:01:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:33 smithi186 ceph-mon[131487]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-17T00:01:34.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:34 smithi006 ceph-mon[97840]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:34.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:35.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:34 smithi087 ceph-mon[131182]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:35.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:34 smithi186 ceph-mon[131487]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:01:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:01:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:35.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:35 smithi006 ceph-mon[97840]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:36.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:35 smithi087 ceph-mon[131182]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:35 smithi186 ceph-mon[131487]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:01:36.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:36 smithi006 ceph-mon[97840]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:37.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:36 smithi087 ceph-mon[131182]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:36 smithi186 ceph-mon[131487]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:37.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]: dispatch 2023-11-17T00:01:37.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]': finished 2023-11-17T00:01:37.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:37 smithi006 ceph-mon[97840]: osdmap e22: 4 total, 3 up, 4 in 2023-11-17T00:01:37.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]: dispatch 2023-11-17T00:01:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]': finished 2023-11-17T00:01:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:37 smithi087 ceph-mon[131182]: osdmap e22: 4 total, 3 up, 4 in 2023-11-17T00:01:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]: dispatch 2023-11-17T00:01:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2851564117' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94de1b8-5e3a-4344-b794-04389000f0d6"}]': finished 2023-11-17T00:01:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:37 smithi186 ceph-mon[131487]: osdmap e22: 4 total, 3 up, 4 in 2023-11-17T00:01:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:38 smithi006 ceph-mon[97840]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-11-17T00:01:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:38 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2651630092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:39.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:38 smithi087 ceph-mon[131182]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-11-17T00:01:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:38 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2651630092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:39.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:38 smithi186 ceph-mon[131487]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-11-17T00:01:39.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:38 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2651630092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:01:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:40 smithi006 ceph-mon[97840]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-17T00:01:41.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:40 smithi087 ceph-mon[131182]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-17T00:01:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:40 smithi186 ceph-mon[131487]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-17T00:01:42.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:42 smithi006 ceph-mon[97840]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-17T00:01:43.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:42 smithi087 ceph-mon[131182]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-17T00:01:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:42 smithi186 ceph-mon[131487]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-17T00:01:44.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:44 smithi006 ceph-mon[97840]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:44.600 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:44 smithi087 ceph-mon[131182]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:44.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:44 smithi186 ceph-mon[131487]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:47.062 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:46 smithi006 ceph-mon[97840]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:47.062 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:46 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-17T00:01:47.063 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:46 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:47.063 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:46 smithi006 ceph-mon[97840]: Deploying daemon osd.3 on smithi006 2023-11-17T00:01:47.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:46 smithi087 ceph-mon[131182]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:47.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:46 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-17T00:01:47.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:46 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:46 smithi087 ceph-mon[131182]: Deploying daemon osd.3 on smithi006 2023-11-17T00:01:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:46 smithi186 ceph-mon[131487]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-17T00:01:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:46 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-17T00:01:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:46 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:46 smithi186 ceph-mon[131487]: Deploying daemon osd.3 on smithi006 2023-11-17T00:01:48.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:48 smithi006 ceph-mon[97840]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:49.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:48 smithi087 ceph-mon[131182]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:48 smithi186 ceph-mon[131487]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:51.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:50 smithi087 ceph-mon[131182]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:50 smithi186 ceph-mon[131487]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:50 smithi006 ceph-mon[97840]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:52 smithi087 ceph-mon[131182]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:52 smithi186 ceph-mon[131487]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:52 smithi006 ceph-mon[97840]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:54.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:54.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:53 smithi006 ceph-mon[97840]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:54.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:54.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:53 smithi087 ceph-mon[131182]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:54.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:53 smithi186 ceph-mon[131487]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:56.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:57.220 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.261 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.291 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:57.291 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:57.292 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.292 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:57.292 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:01:57.293 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:57.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.245+0000 7f94837fe700 1 -- 172.21.15.6:0/1051281938 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9494060f30 con 0x7f947c0a1150 2023-11-17T00:01:57.294 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 3 on host 'smithi006' 2023-11-17T00:01:57.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 msgr2=0x7f947c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:57.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 0x7f947c0a3600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f948400b2e0 tx=0x7f9484005fb0 comp rx=0 tx=0).stop 2023-11-17T00:01:57.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 msgr2=0x7f94941986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:01:57.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948800eb60 tx=0x7f948800ee70 comp rx=0 tx=0).stop 2023-11-17T00:01:57.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 shutdown_connections 2023-11-17T00:01:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f947c0a1150 0x7f947c0a3600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9494108230 0x7f949419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940734d0 0x7f94941981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 --2- 172.21.15.6:0/1051281938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9494074aa0 0x7f94941986f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:01:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 >> 172.21.15.6:0/1051281938 conn(0x7f94940fb9a0 msgr2=0x7f94940fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:01:57.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 shutdown_connections 2023-11-17T00:01:57.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:01:57.248+0000 7f9498cc0700 1 -- 172.21.15.6:0/1051281938 wait complete. 2023-11-17T00:01:57.758 DEBUG:teuthology.orchestra.run.smithi006:osd.3> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.3.service 2023-11-17T00:01:57.762 INFO:tasks.cephadm:Deploying osd.4 on smithi087 with /dev/vg_nvme/lv_3... 2023-11-17T00:01:57.763 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-17T00:01:57.838 INFO:journalctl@ceph.osd.3.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:01:58.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:57 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:58.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:01:58.143 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-17T00:01:58.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:57 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:58.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:01:58.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:01:58.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-17T00:01:58.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:01:58.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:57 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:01:58.406 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:01:58.906 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:58 smithi087 ceph-mon[131182]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:58.906 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:58 smithi087 ceph-mon[131182]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-17T00:01:58.906 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:58 smithi087 ceph-mon[131182]: osdmap e23: 4 total, 3 up, 4 in 2023-11-17T00:01:58.906 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:58.906 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:58 smithi087 ceph-mon[131182]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:01:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:58 smithi186 ceph-mon[131487]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:58 smithi186 ceph-mon[131487]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-17T00:01:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:58 smithi186 ceph-mon[131487]: osdmap e23: 4 total, 3 up, 4 in 2023-11-17T00:01:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:58 smithi186 ceph-mon[131487]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:01:59.192 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-mon[97840]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:01:59.192 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-mon[97840]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-17T00:01:59.192 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-mon[97840]: osdmap e23: 4 total, 3 up, 4 in 2023-11-17T00:01:59.192 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:59.192 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-mon[97840]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:01:59.192 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:01:58.762+0000 7fa92b9a5700 -1 osd.3 0 waiting for initial osdmap 2023-11-17T00:01:59.192 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:01:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:01:58.772+0000 7fa92779a700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:01:59.904 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-17T00:01:59.904 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-17T00:01:59.904 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-11-17T00:01:59.904 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-11-17T00:01:59.905 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012456 s, 842 MB/s 2023-11-17T00:01:59.905 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: osdmap e24: 4 total, 3 up, 4 in 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] boot 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: osdmap e25: 4 total, 4 up, 4 in 2023-11-17T00:01:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:01:59 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.023 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:02:00.023 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: osdmap e24: 4 total, 3 up, 4 in 2023-11-17T00:02:00.023 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.023 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.024 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] boot 2023-11-17T00:02:00.024 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: osdmap e25: 4 total, 4 up, 4 in 2023-11-17T00:02:00.024 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:01:59 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: from='osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: osdmap e24: 4 total, 3 up, 4 in 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: osd.3 [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] boot 2023-11-17T00:02:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: osdmap e25: 4 total, 4 up, 4 in 2023-11-17T00:02:00.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:01:59 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:02:00.245 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_3 2023-11-17T00:02:00.883 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:02:01.078 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:00 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:02:01.078 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:00 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:02:01.078 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:00 smithi087 ceph-mon[131182]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:02:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:00 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:02:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:00 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:02:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:00 smithi186 ceph-mon[131487]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:02:01.118 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:00 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:02:01.118 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:00 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:02:01.118 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:00 smithi006 ceph-mon[97840]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1801469104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c100d70 msgr2=0x7f7f5c107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1801469104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c100d70 0x7f7f5c107630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b0a0 tx=0x7f7f5400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1801469104 shutdown_connections 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1801469104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c100d70 0x7f7f5c107630 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1801469104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 0x7f7f5c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1801469104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1801469104 >> 172.21.15.87:0/1801469104 conn(0x7f7f5c0fa7f0 msgr2=0x7f7f5c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1801469104 shutdown_connections 2023-11-17T00:02:01.951 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1801469104 wait complete. 2023-11-17T00:02:01.953 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 Processor -- start 2023-11-17T00:02:01.953 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- start start 2023-11-17T00:02:01.953 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 0x7f7f5c1950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 0x7f7f5c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19cc30 con 0x7f7f5c100d70 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19cd70 con 0x7f7f5c1003e0 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f62ff7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19cf00 con 0x7f7f5c0ff1e0 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.950+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35464/0 (socket says 172.21.15.87:35464) 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 -- 172.21.15.87:0/1995975753 learned_addr learned my addr 172.21.15.87:0/1995975753 (peer_addr_for_me v2:172.21.15.87:0/0) 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f60d93700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 0x7f7f5c1950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f61594700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 0x7f7f5c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 -- 172.21.15.87:0/1995975753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 msgr2=0x7f7f5c1950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:01.954 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 0x7f7f5c1950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 -- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 msgr2=0x7f7f5c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 0x7f7f5c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 -- 172.21.15.87:0/1995975753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54009d00 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f61594700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 0x7f7f5c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f5bfff700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d950 tx=0x7f7f4800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48009940 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c10ab90 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f48010460 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800f600 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.951+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c10aa00 con 0x7f7f5c1003e0 2023-11-17T00:02:01.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.952+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c19d630 con 0x7f7f5c1003e0 2023-11-17T00:02:01.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.953+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7f480105d0 con 0x7f7f5c1003e0 2023-11-17T00:02:01.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.954+0000 7f7f59ffb700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 0x7f7f4c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:01.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.954+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f7f480cb4f0 con 0x7f7f5c1003e0 2023-11-17T00:02:01.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.954+0000 7f7f60d93700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 0x7f7f4c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:01.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.955+0000 7f7f60d93700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 0x7f7f4c0a3680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c100240 tx=0x7f7f50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:01.959 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:01.958+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7f48095e40 con 0x7f7f5c1003e0 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: osdmap e26: 4 total, 4 up, 4 in 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: osdmap e26: 4 total, 4 up, 4 in 2023-11-17T00:02:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:02.152+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f5c19dbe0 con 0x7f7f4c0a11d0 2023-11-17T00:02:02.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: osdmap e26: 4 total, 4 up, 4 in 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:02.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:03.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi006 2023-11-17T00:02:03.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 8638M 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: osdmap e27: 4 total, 4 up, 4 in 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:03.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi006 2023-11-17T00:02:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 8638M 2023-11-17T00:02:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: osdmap e27: 4 total, 4 up, 4 in 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:03.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi006 2023-11-17T00:02:03.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 8638M 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: osdmap e27: 4 total, 4 up, 4 in 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:03 smithi087 ceph-mon[131182]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:03 smithi087 ceph-mon[131182]: osdmap e28: 4 total, 4 up, 4 in 2023-11-17T00:02:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:03 smithi186 ceph-mon[131487]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:03 smithi186 ceph-mon[131487]: osdmap e28: 4 total, 4 up, 4 in 2023-11-17T00:02:04.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:03 smithi006 ceph-mon[97840]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:03 smithi006 ceph-mon[97840]: osdmap e28: 4 total, 4 up, 4 in 2023-11-17T00:02:05.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: osdmap e29: 4 total, 4 up, 4 in 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/3268855288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]': finished 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: osdmap e30: 5 total, 4 up, 5 in 2023-11-17T00:02:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: osdmap e29: 4 total, 4 up, 4 in 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/3268855288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]': finished 2023-11-17T00:02:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: osdmap e30: 5 total, 4 up, 5 in 2023-11-17T00:02:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:05.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:05.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: osdmap e29: 4 total, 4 up, 4 in 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/3268855288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]: dispatch 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17ec1e9b-999c-47e6-87ca-3405277892f1"}]': finished 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: osdmap e30: 5 total, 4 up, 5 in 2023-11-17T00:02:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:06.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:05 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/563836821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:06.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:05 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/563836821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:06.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:05 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/563836821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:07.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:06 smithi087 ceph-mon[131182]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:07.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:06 smithi186 ceph-mon[131487]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:07.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:06 smithi006 ceph-mon[97840]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-17T00:02:08.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:07 smithi087 ceph-mon[131182]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-17T00:02:08.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:07 smithi087 ceph-mon[131182]: Cluster is now healthy 2023-11-17T00:02:08.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:07 smithi186 ceph-mon[131487]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-17T00:02:08.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:07 smithi186 ceph-mon[131487]: Cluster is now healthy 2023-11-17T00:02:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:07 smithi006 ceph-mon[97840]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-17T00:02:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:07 smithi006 ceph-mon[97840]: Cluster is now healthy 2023-11-17T00:02:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:08 smithi087 ceph-mon[131182]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-17T00:02:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:08 smithi186 ceph-mon[131487]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-17T00:02:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:08 smithi006 ceph-mon[97840]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-17T00:02:11.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:10 smithi087 ceph-mon[131182]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-17T00:02:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:10 smithi186 ceph-mon[131487]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-17T00:02:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:10 smithi006 ceph-mon[97840]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-17T00:02:12.022 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-17T00:02:12.022 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:12.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-17T00:02:12.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-17T00:02:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:13.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:12 smithi087 ceph-mon[131182]: Deploying daemon osd.4 on smithi087 2023-11-17T00:02:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:12 smithi087 ceph-mon[131182]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:02:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:12 smithi186 ceph-mon[131487]: Deploying daemon osd.4 on smithi087 2023-11-17T00:02:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:12 smithi186 ceph-mon[131487]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:02:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:12 smithi006 ceph-mon[97840]: Deploying daemon osd.4 on smithi087 2023-11-17T00:02:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:12 smithi006 ceph-mon[97840]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:02:14.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:14 smithi087 ceph-mon[131182]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-17T00:02:14.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:14 smithi186 ceph-mon[131487]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-17T00:02:14.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:14 smithi006 ceph-mon[97840]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-17T00:02:16.470 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.471 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.471 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:16 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:16.471 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:16 smithi087 ceph-mon[131182]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-17T00:02:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:16 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:16 smithi186 ceph-mon[131487]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-17T00:02:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:16 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:16 smithi006 ceph-mon[97840]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-17T00:02:17.969 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 4 on host 'smithi087' 2023-11-17T00:02:17.970 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.968+0000 7f7f59ffb700 1 -- 172.21.15.87:0/1995975753 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7f5c19dbe0 con 0x7f7f4c0a11d0 2023-11-17T00:02:17.973 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 msgr2=0x7f7f4c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:17.973 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 0x7f7f4c0a3680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c100240 tx=0x7f7f50009260 comp rx=0 tx=0).stop 2023-11-17T00:02:17.973 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 msgr2=0x7f7f5c1955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:17.973 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d950 tx=0x7f7f4800dc60 comp rx=0 tx=0).stop 2023-11-17T00:02:17.974 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 shutdown_connections 2023-11-17T00:02:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7f4c0a11d0 0x7f7f4c0a3680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:17.975 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f5c100d70 0x7f7f5c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ff1e0 0x7f7f5c1950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:17.976 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 --2- 172.21.15.87:0/1995975753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f5c1003e0 0x7f7f5c1955e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 >> 172.21.15.87:0/1995975753 conn(0x7f7f5c0fa7f0 msgr2=0x7f7f5c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:02:17.977 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 shutdown_connections 2023-11-17T00:02:17.978 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:02:17.970+0000 7f7f62ff7700 1 -- 172.21.15.87:0/1995975753 wait complete. 2023-11-17T00:02:18.358 DEBUG:teuthology.orchestra.run.smithi087:osd.4> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.4.service 2023-11-17T00:02:18.360 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2023-11-17T00:02:18.361 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-17T00:02:18.403 INFO:journalctl@ceph.osd.4.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:02:18.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:02:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:02:18.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:18.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:02:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:18.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:19.026 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:02:19.499 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:02:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:02:19.278+0000 7f04f41b1640 -1 osd.4 0 log_to_monitors true 2023-11-17T00:02:19.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:19 smithi186 ceph-mon[131487]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:19.614 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:19 smithi186 ceph-mon[131487]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:19.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:19 smithi087 ceph-mon[131182]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:19.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:19 smithi087 ceph-mon[131182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:19.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:19 smithi006 ceph-mon[97840]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:19.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:19 smithi006 ceph-mon[97840]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-17T00:02:20.623 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-17T00:02:20.624 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-17T00:02:20.624 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-11-17T00:02:20.624 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-11-17T00:02:20.624 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109929 s, 954 MB/s 2023-11-17T00:02:20.624 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: osdmap e31: 5 total, 4 up, 5 in 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.772 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:20 smithi186 ceph-mon[131487]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:20.851 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:02:20.515+0000 7f04e9828700 -1 osd.4 0 waiting for initial osdmap 2023-11-17T00:02:20.851 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:02:20.521+0000 7f04e561d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:02:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-17T00:02:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: osdmap e31: 5 total, 4 up, 5 in 2023-11-17T00:02:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:20 smithi087 ceph-mon[131182]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:20.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-17T00:02:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: osdmap e31: 5 total, 4 up, 5 in 2023-11-17T00:02:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: from='osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:02:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:20 smithi006 ceph-mon[97840]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:21.077 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2023-11-17T00:02:21.717 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: osdmap e32: 5 total, 4 up, 5 in 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi087 2023-11-17T00:02:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 8584M 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:02:21.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:21 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:21.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:02:21.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: osdmap e32: 5 total, 4 up, 5 in 2023-11-17T00:02:21.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi087 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 8584M 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:02:21.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:21 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: osdmap e32: 5 total, 4 up, 5 in 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi087 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 8584M 2023-11-17T00:02:21.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:02:21.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:21 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:22.540 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] boot 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: osdmap e33: 5 total, 5 up, 5 in 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.541 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:22 smithi186 ceph-mon[131487]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:02:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:02:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] boot 2023-11-17T00:02:22.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: osdmap e33: 5 total, 5 up, 5 in 2023-11-17T00:02:22.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:22 smithi087 ceph-mon[131182]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: osd.4 [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] boot 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: osdmap e33: 5 total, 5 up, 5 in 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:02:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:22 smithi006 ceph-mon[97840]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-17T00:02:23.089 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 -- 172.21.15.186:0/4171949677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9080ff1e0 msgr2=0x7fa9080ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:23.089 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 --2- 172.21.15.186:0/4171949677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9080ff1e0 0x7fa9080ff5f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009990 tx=0x7fa8f8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.089 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 -- 172.21.15.186:0/4171949677 shutdown_connections 2023-11-17T00:02:23.089 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 --2- 172.21.15.186:0/4171949677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908100d70 0x7fa908107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 --2- 172.21.15.186:0/4171949677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9080ff1e0 0x7fa9080ff5f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 --2- 172.21.15.186:0/4171949677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9081003e0 0x7fa908100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 -- 172.21.15.186:0/4171949677 >> 172.21.15.186:0/4171949677 conn(0x7fa9080fa7f0 msgr2=0x7fa9080fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 -- 172.21.15.186:0/4171949677 shutdown_connections 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.085+0000 7fa90e71a700 1 -- 172.21.15.186:0/4171949677 wait complete. 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 Processor -- start 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 -- start start 2023-11-17T00:02:23.090 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080ff1e0 0x7fa908195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:23.091 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9081003e0 0x7fa908195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:23.091 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:23.091 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90819cb90 con 0x7fa9081003e0 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90819ccd0 con 0x7fa9080ff1e0 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90e71a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908068ec0 con 0x7fa908100d70 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90ccb7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90ccb7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55690/0 (socket says 172.21.15.186:55690) 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.086+0000 7fa90ccb7700 1 -- 172.21.15.186:0/1726117170 learned_addr learned my addr 172.21.15.186:0/1726117170 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa907fff700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080ff1e0 0x7fa908195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 -- 172.21.15.186:0/1726117170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080ff1e0 msgr2=0x7fa908195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080ff1e0 0x7fa908195000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.092 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 -- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9081003e0 msgr2=0x7fa908195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9081003e0 0x7fa908195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 -- 172.21.15.186:0/1726117170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80095f0 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90ccb7700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b970 tx=0x7fa90000bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000c9b0 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa900014440 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90810ac30 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000075d0 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.087+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90810aaa0 con 0x7fa908100d70 2023-11-17T00:02:23.093 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.088+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908066ea0 con 0x7fa908100d70 2023-11-17T00:02:23.094 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.089+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa9000145b0 con 0x7fa908100d70 2023-11-17T00:02:23.094 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.090+0000 7fa9057fa700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 0x7fa8f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:23.094 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.090+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fa900006e40 con 0x7fa908100d70 2023-11-17T00:02:23.094 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.090+0000 7fa907fff700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 0x7fa8f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:23.095 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.091+0000 7fa907fff700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 0x7fa8f00a3600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80095c0 tx=0x7fa8f8009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:23.095 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.094+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa90004e730 con 0x7fa908100d70 2023-11-17T00:02:23.293 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:23.291+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa908069680 con 0x7fa8f00a1150 2023-11-17T00:02:23.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:23 smithi087 ceph-mon[131182]: osdmap e34: 5 total, 5 up, 5 in 2023-11-17T00:02:23.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:23.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:23.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:23.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:23 smithi186 ceph-mon[131487]: osdmap e34: 5 total, 5 up, 5 in 2023-11-17T00:02:23.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:23.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:23.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:23.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:23 smithi006 ceph-mon[97840]: osdmap e34: 5 total, 5 up, 5 in 2023-11-17T00:02:23.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:23.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:23.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:24.805 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:24 smithi186 ceph-mon[131487]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:24.805 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:24 smithi186 ceph-mon[131487]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:24.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:24 smithi087 ceph-mon[131182]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:24.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:24 smithi087 ceph-mon[131182]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:24.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:24 smithi006 ceph-mon[97840]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:24.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:24 smithi006 ceph-mon[97840]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/2972423192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]': finished 2023-11-17T00:02:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: osdmap e35: 6 total, 5 up, 6 in 2023-11-17T00:02:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:26 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/420580348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:27.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:27.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/2972423192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]': finished 2023-11-17T00:02:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: osdmap e35: 6 total, 5 up, 6 in 2023-11-17T00:02:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:26 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/420580348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/2972423192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]: dispatch 2023-11-17T00:02:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e5ab3b-1735-4f77-82c8-a374f7fab368"}]': finished 2023-11-17T00:02:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: osdmap e35: 6 total, 5 up, 6 in 2023-11-17T00:02:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:26 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/420580348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:29.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:28 smithi087 ceph-mon[131182]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:28 smithi186 ceph-mon[131487]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:29.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:28 smithi006 ceph-mon[97840]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:30 smithi087 ceph-mon[131182]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:31.102 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:30 smithi186 ceph-mon[131487]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:31.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:30 smithi006 ceph-mon[97840]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:33.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:32 smithi087 ceph-mon[131182]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-17T00:02:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:32 smithi186 ceph-mon[131487]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-17T00:02:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:33.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:32 smithi006 ceph-mon[97840]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-17T00:02:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:33 smithi087 ceph-mon[131182]: Deploying daemon osd.5 on smithi186 2023-11-17T00:02:34.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:33 smithi186 ceph-mon[131487]: Deploying daemon osd.5 on smithi186 2023-11-17T00:02:34.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:33 smithi006 ceph-mon[97840]: Deploying daemon osd.5 on smithi186 2023-11-17T00:02:35.019 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:34 smithi186 ceph-mon[131487]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:34 smithi087 ceph-mon[131182]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:34 smithi006 ceph-mon[97840]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:37.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:36 smithi087 ceph-mon[131182]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:37.105 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:36 smithi186 ceph-mon[131487]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:36 smithi006 ceph-mon[97840]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:38.494 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.494 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.494 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:38.494 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:38 smithi186 ceph-mon[131487]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:38.600 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:38.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:38 smithi087 ceph-mon[131182]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:38.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:38.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:38.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:38 smithi006 ceph-mon[97840]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:39.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.960+0000 7fa9057fa700 1 -- 172.21.15.186:0/1726117170 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa908069680 con 0x7fa8f00a1150 2023-11-17T00:02:39.962 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2023-11-17T00:02:39.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 msgr2=0x7fa8f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 0x7fa8f00a3600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80095c0 tx=0x7fa8f8009530 comp rx=0 tx=0).stop 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 msgr2=0x7fa90819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b970 tx=0x7fa90000bd30 comp rx=0 tx=0).stop 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 shutdown_connections 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa8f00a1150 0x7fa8f00a3600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa908100d70 0x7fa90819c5c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080ff1e0 0x7fa908195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:39.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 --2- 172.21.15.186:0/1726117170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9081003e0 0x7fa908195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:39.966 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 >> 172.21.15.186:0/1726117170 conn(0x7fa9080fa7f0 msgr2=0x7fa908103920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:02:39.966 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 shutdown_connections 2023-11-17T00:02:39.966 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:02:39.963+0000 7fa90e71a700 1 -- 172.21.15.186:0/1726117170 wait complete. 2023-11-17T00:02:40.410 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.5.service 2023-11-17T00:02:40.412 INFO:tasks.cephadm:Deploying osd.6 on smithi006 with /dev/vg_nvme/lv_2... 2023-11-17T00:02:40.412 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-17T00:02:40.442 INFO:journalctl@ceph.osd.5.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:02:41.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:40 smithi006 ceph-mon[97840]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:41.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.243 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:40 smithi186 ceph-mon[131487]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:41.243 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.243 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.244 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.244 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:02:41.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:40 smithi087 ceph-mon[131182]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.362 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:41.599 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:02:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:02:41.241+0000 7f79b7b91640 -1 osd.5 0 log_to_monitors true 2023-11-17T00:02:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:41 smithi006 ceph-mon[97840]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:41 smithi006 ceph-mon[97840]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:42.267 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:41 smithi186 ceph-mon[131487]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:42.267 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:41 smithi186 ceph-mon[131487]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:42.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:41 smithi087 ceph-mon[131182]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:41 smithi087 ceph-mon[131182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-17T00:02:43.115 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:02:42.951+0000 7f79ad208700 -1 osd.5 0 waiting for initial osdmap 2023-11-17T00:02:43.116 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:02:42.960+0000 7f79a8ffd700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: osdmap e36: 6 total, 5 up, 6 in 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:42 smithi186 ceph-mon[131487]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: osdmap e36: 6 total, 5 up, 6 in 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:42 smithi006 ceph-mon[97840]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:43.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-17T00:02:43.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: osdmap e36: 6 total, 5 up, 6 in 2023-11-17T00:02:43.357 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:43.359 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: from='osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.359 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:42 smithi087 ceph-mon[131182]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0316153 s, 332 MB/s 2023-11-17T00:02:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2023-11-17T00:02:44.193 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2023-11-17T00:02:44.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:02:44.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: osdmap e37: 6 total, 5 up, 6 in 2023-11-17T00:02:44.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] boot 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: osdmap e38: 6 total, 6 up, 6 in 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: osdmap e37: 6 total, 5 up, 6 in 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] boot 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: osdmap e38: 6 total, 6 up, 6 in 2023-11-17T00:02:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:02:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: osdmap e37: 6 total, 5 up, 6 in 2023-11-17T00:02:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: osd.5 [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] boot 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: osdmap e38: 6 total, 6 up, 6 in 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:02:44.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:44.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:02:44.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:02:45.054 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:02:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:02:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi186 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 8584M 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:44 smithi006 ceph-mon[97840]: osdmap e39: 6 total, 6 up, 6 in 2023-11-17T00:02:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:02:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:02:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi186 2023-11-17T00:02:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 8584M 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:45.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:44 smithi087 ceph-mon[131182]: osdmap e39: 6 total, 6 up, 6 in 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi186 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 8584M 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:02:45.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:02:45.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-17T00:02:45.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:44 smithi186 ceph-mon[131487]: osdmap e39: 6 total, 6 up, 6 in 2023-11-17T00:02:46.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:46 smithi006 ceph-mon[97840]: osdmap e40: 6 total, 6 up, 6 in 2023-11-17T00:02:46.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:46 smithi006 ceph-mon[97840]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:46.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:46 smithi087 ceph-mon[131182]: osdmap e40: 6 total, 6 up, 6 in 2023-11-17T00:02:46.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:46 smithi087 ceph-mon[131182]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:46.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:46 smithi186 ceph-mon[131487]: osdmap e40: 6 total, 6 up, 6 in 2023-11-17T00:02:46.618 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:46 smithi186 ceph-mon[131487]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 -- 172.21.15.6:0/3664880616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481014f0 msgr2=0x7f6c48101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/3664880616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481014f0 0x7f6c48101940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009990 tx=0x7f6c3c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 -- 172.21.15.6:0/3664880616 shutdown_connections 2023-11-17T00:02:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/3664880616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 0x7f6c48108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/3664880616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481002f0 0x7f6c48100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/3664880616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481014f0 0x7f6c48101940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 -- 172.21.15.6:0/3664880616 >> 172.21.15.6:0/3664880616 conn(0x7f6c480fb900 msgr2=0x7f6c480fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:02:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 -- 172.21.15.6:0/3664880616 shutdown_connections 2023-11-17T00:02:46.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.898+0000 7f6c4c81a700 1 -- 172.21.15.6:0/3664880616 wait complete. 2023-11-17T00:02:46.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 Processor -- start 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 -- start start 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481014f0 0x7f6c48198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 0x7f6c4819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.899+0000 7f6c4c81a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4819fd90 con 0x7f6c481002f0 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4c81a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4819fed0 con 0x7f6c48101e80 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4c81a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481a0060 con 0x7f6c481014f0 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47718/0 (socket says 172.21.15.6:47718) 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 -- 172.21.15.6:0/794084895 learned_addr learned my addr 172.21.15.6:0/794084895 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c46d9d700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 0x7f6c4819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:46.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c45d9b700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481014f0 0x7f6c48198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 -- 172.21.15.6:0/794084895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481014f0 msgr2=0x7f6c48198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481014f0 0x7f6c48198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 -- 172.21.15.6:0/794084895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 msgr2=0x7f6c4819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 0x7f6c4819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 -- 172.21.15.6:0/794084895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0095f0 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.900+0000 7f6c4659c700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800e9a0 tx=0x7f6c3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.901+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38009790 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.901+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c3800c810 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.901+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003c70 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.901+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4810bc50 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.901+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4810bac0 con 0x7f6c481002f0 2023-11-17T00:02:46.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.902+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c481929b0 con 0x7f6c481002f0 2023-11-17T00:02:46.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.904+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6c3800cc80 con 0x7f6c481002f0 2023-11-17T00:02:46.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.907+0000 7f6c377fe700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 0x7f6c300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:02:46.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.907+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f6c38014070 con 0x7f6c481002f0 2023-11-17T00:02:46.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.907+0000 7f6c45d9b700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 0x7f6c300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:02:46.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.907+0000 7f6c45d9b700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 0x7f6c300a3680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00b680 tx=0x7f6c3c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:02:46.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:46.909+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6c38096230 con 0x7f6c481002f0 2023-11-17T00:02:47.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:02:47.124+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c48061c40 con 0x7f6c300a11d0 2023-11-17T00:02:47.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:47 smithi087 ceph-mon[131182]: osdmap e41: 6 total, 6 up, 6 in 2023-11-17T00:02:47.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:47.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:47.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:47 smithi186 ceph-mon[131487]: osdmap e41: 6 total, 6 up, 6 in 2023-11-17T00:02:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:47.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:47.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:47 smithi006 ceph-mon[97840]: osdmap e41: 6 total, 6 up, 6 in 2023-11-17T00:02:47.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:02:47.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:02:47.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:48.600 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:48 smithi087 ceph-mon[131182]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:48.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:48 smithi087 ceph-mon[131182]: osdmap e42: 6 total, 6 up, 6 in 2023-11-17T00:02:48.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:48 smithi087 ceph-mon[131182]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:02:48.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:48 smithi186 ceph-mon[131487]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:48.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:48 smithi186 ceph-mon[131487]: osdmap e42: 6 total, 6 up, 6 in 2023-11-17T00:02:48.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:48 smithi186 ceph-mon[131487]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:02:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:48 smithi006 ceph-mon[97840]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:02:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:48 smithi006 ceph-mon[97840]: osdmap e42: 6 total, 6 up, 6 in 2023-11-17T00:02:48.698 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:48 smithi006 ceph-mon[97840]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:02:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:50 smithi087 ceph-mon[131182]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:02:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:50 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]: dispatch 2023-11-17T00:02:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:50 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]': finished 2023-11-17T00:02:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:50 smithi087 ceph-mon[131182]: osdmap e43: 7 total, 6 up, 7 in 2023-11-17T00:02:51.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:02:51.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:50 smithi186 ceph-mon[131487]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:02:51.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:50 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]: dispatch 2023-11-17T00:02:51.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:50 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]': finished 2023-11-17T00:02:51.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:50 smithi186 ceph-mon[131487]: osdmap e43: 7 total, 6 up, 7 in 2023-11-17T00:02:51.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:02:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:50 smithi006 ceph-mon[97840]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:02:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:50 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]: dispatch 2023-11-17T00:02:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:50 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3766904781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8efcc3e3-0f96-4248-8eac-6f5041c0b4fb"}]': finished 2023-11-17T00:02:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:50 smithi006 ceph-mon[97840]: osdmap e43: 7 total, 6 up, 7 in 2023-11-17T00:02:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:02:52.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:51 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3810539289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:52.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:51 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3810539289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:52.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:51 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3810539289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:02:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:52 smithi087 ceph-mon[131182]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-17T00:02:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:52 smithi186 ceph-mon[131487]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-17T00:02:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:52 smithi006 ceph-mon[97840]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-17T00:02:54.600 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:54 smithi087 ceph-mon[131182]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-11-17T00:02:54.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:54 smithi186 ceph-mon[131487]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-11-17T00:02:54.649 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:54 smithi006 ceph-mon[97840]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-11-17T00:02:56.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:56 smithi006 ceph-mon[97840]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:56 smithi087 ceph-mon[131182]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:57.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:56 smithi186 ceph-mon[131487]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:58.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:58 smithi006 ceph-mon[97840]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-17T00:02:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:58 smithi087 ceph-mon[131182]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-17T00:02:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:02:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:58 smithi186 ceph-mon[131487]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:02:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-17T00:02:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:00.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:02:59 smithi006 ceph-mon[97840]: Deploying daemon osd.6 on smithi006 2023-11-17T00:03:00.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:02:59 smithi087 ceph-mon[131182]: Deploying daemon osd.6 on smithi006 2023-11-17T00:03:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:02:59 smithi186 ceph-mon[131487]: Deploying daemon osd.6 on smithi006 2023-11-17T00:03:01.011 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:00 smithi006 ceph-mon[97840]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:00 smithi087 ceph-mon[131182]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:00 smithi186 ceph-mon[131487]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:02 smithi006 ceph-mon[97840]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:03.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:02 smithi087 ceph-mon[131182]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:02 smithi186 ceph-mon[131487]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:04.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:04 smithi006 ceph-mon[97840]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:04.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:04 smithi087 ceph-mon[131182]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:04.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:04 smithi186 ceph-mon[131487]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:06.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:06.922 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:06.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:06.923 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:06 smithi006 ceph-mon[97840]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:06 smithi087 ceph-mon[131182]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:06 smithi186 ceph-mon[131487]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:08 smithi087 ceph-mon[131182]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:08 smithi186 ceph-mon[131487]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:08 smithi006 ceph-mon[97840]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:09.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:09.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:09 smithi006 ceph-mon[97840]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-17T00:03:09.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.989+0000 7f6c377fe700 1 -- 172.21.15.6:0/794084895 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c48061c40 con 0x7f6c300a11d0 2023-11-17T00:03:09.992 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 6 on host 'smithi006' 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 msgr2=0x7f6c300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 0x7f6c300a3680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00b680 tx=0x7f6c3c005fd0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 msgr2=0x7f6c48198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800e9a0 tx=0x7f6c3800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 shutdown_connections 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f6c300a11d0 0x7f6c300a3680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c481002f0 0x7f6c48198200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c48101e80 0x7f6c4819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 --2- 172.21.15.6:0/794084895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c481014f0 0x7f6c48198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:09.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 >> 172.21.15.6:0/794084895 conn(0x7f6c480fb900 msgr2=0x7f6c48106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:03:09.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 shutdown_connections 2023-11-17T00:03:09.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:03:09.992+0000 7f6c4c81a700 1 -- 172.21.15.6:0/794084895 wait complete. 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:10.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:03:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:09 smithi087 ceph-mon[131182]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:03:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:09 smithi186 ceph-mon[131487]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-17T00:03:10.597 DEBUG:teuthology.orchestra.run.smithi006:osd.6> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.6.service 2023-11-17T00:03:10.642 INFO:tasks.cephadm:Deploying osd.7 on smithi087 with /dev/vg_nvme/lv_2... 2023-11-17T00:03:10.647 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-17T00:03:10.715 INFO:journalctl@ceph.osd.6.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: osdmap e44: 7 total, 6 up, 7 in 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:10 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:11.222 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:03:10.898+0000 7f2020e32700 -1 osd.6 0 waiting for initial osdmap 2023-11-17T00:03:11.222 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:03:10.914+0000 7f201b424700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: osdmap e44: 7 total, 6 up, 7 in 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:10 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:11.296 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:03:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-17T00:03:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: osdmap e44: 7 total, 6 up, 7 in 2023-11-17T00:03:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:03:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:10 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:11 smithi087 ceph-mon[131182]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:03:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:11 smithi087 ceph-mon[131182]: osdmap e45: 7 total, 6 up, 7 in 2023-11-17T00:03:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:11 smithi006 ceph-mon[97840]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:03:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:11 smithi006 ceph-mon[97840]: osdmap e45: 7 total, 6 up, 7 in 2023-11-17T00:03:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:11 smithi186 ceph-mon[131487]: from='osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:03:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:11 smithi186 ceph-mon[131487]: osdmap e45: 7 total, 6 up, 7 in 2023-11-17T00:03:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:12.732 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-17T00:03:12.732 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-17T00:03:12.732 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-11-17T00:03:12.733 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-11-17T00:03:12.733 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113033 s, 928 MB/s 2023-11-17T00:03:12.733 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-11-17T00:03:13.088 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:03:13.088 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:03:13.088 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:13.088 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.089 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] boot 2023-11-17T00:03:13.089 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: osdmap e46: 7 total, 7 up, 7 in 2023-11-17T00:03:13.089 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.147 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_2 2023-11-17T00:03:13.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] boot 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: osdmap e46: 7 total, 7 up, 7 in 2023-11-17T00:03:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:03:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:03:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-17T00:03:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: osd.6 [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] boot 2023-11-17T00:03:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: osdmap e46: 7 total, 7 up, 7 in 2023-11-17T00:03:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:03:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:03:13.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: osdmap e47: 7 total, 7 up, 7 in 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:13.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:13 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: osdmap e47: 7 total, 7 up, 7 in 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.168 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:14.169 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:14.169 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:13 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: osdmap e47: 7 total, 7 up, 7 in 2023-11-17T00:03:14.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:14.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:14.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:14.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:13 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.909+0000 7fd567da7700 1 -- 172.21.15.87:0/2832026306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ffde0 msgr2=0x7fd560107690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.909+0000 7fd567da7700 1 --2- 172.21.15.87:0/2832026306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ffde0 0x7fd560107690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b0a0 tx=0x7fd55800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 -- 172.21.15.87:0/2832026306 shutdown_connections 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 --2- 172.21.15.87:0/2832026306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ffde0 0x7fd560107690 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 --2- 172.21.15.87:0/2832026306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560100860 0x7fd560100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.913 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 --2- 172.21.15.87:0/2832026306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5600ff430 0x7fd5600ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 -- 172.21.15.87:0/2832026306 >> 172.21.15.87:0/2832026306 conn(0x7fd5600f9f70 msgr2=0x7fd5600fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:03:14.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 -- 172.21.15.87:0/2832026306 shutdown_connections 2023-11-17T00:03:14.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 -- 172.21.15.87:0/2832026306 wait complete. 2023-11-17T00:03:14.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.910+0000 7fd567da7700 1 Processor -- start 2023-11-17T00:03:14.914 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 -- start start 2023-11-17T00:03:14.955 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:14.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5600ffde0 0x7fd560195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:14.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560100860 0x7fd56019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:14.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019cb50 con 0x7fd5600ffde0 2023-11-17T00:03:14.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019cc90 con 0x7fd5600ff430 2023-11-17T00:03:14.956 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019ce20 con 0x7fd560100860 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35572/0 (socket says 172.21.15.87:35572) 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 -- 172.21.15.87:0/3429187651 learned_addr learned my addr 172.21.15.87:0/3429187651 (peer_addr_for_me v2:172.21.15.87:0/0) 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 -- 172.21.15.87:0/3429187651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560100860 msgr2=0x7fd56019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560100860 0x7fd56019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.957 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 -- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5600ffde0 msgr2=0x7fd560195500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:14.958 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5600ffde0 0x7fd560195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:14.958 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 -- 172.21.15.87:0/3429187651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558009d00 con 0x7fd5600ff430 2023-11-17T00:03:14.958 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd565b43700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c920 tx=0x7fd55400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:03:14.958 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554007b00 con 0x7fd5600ff430 2023-11-17T00:03:14.958 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56010abf0 con 0x7fd5600ff430 2023-11-17T00:03:14.959 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd554004cf0 con 0x7fd5600ff430 2023-11-17T00:03:14.959 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.911+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554005520 con 0x7fd5600ff430 2023-11-17T00:03:14.959 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.912+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56010aa60 con 0x7fd5600ff430 2023-11-17T00:03:14.961 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.913+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600fe720 con 0x7fd5600ff430 2023-11-17T00:03:14.961 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.914+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd554004750 con 0x7fd5600ff430 2023-11-17T00:03:14.961 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.914+0000 7fd552ffd700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 0x7fd54c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:14.961 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.914+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fd5540cb8f0 con 0x7fd5600ff430 2023-11-17T00:03:14.962 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.914+0000 7fd565342700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 0x7fd54c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:14.962 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.915+0000 7fd565342700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 0x7fd54c0a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005950 tx=0x7fd55c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:03:14.962 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:14.918+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd554095d50 con 0x7fd5600ff430 2023-11-17T00:03:15.113 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:15.112+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd560060f30 con 0x7fd54c0a1150 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi006 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 5758M 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:03:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:14 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:15.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi006 2023-11-17T00:03:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 5758M 2023-11-17T00:03:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:03:15.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:14 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:15.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:15.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi006 2023-11-17T00:03:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 5758M 2023-11-17T00:03:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:03:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:14 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:16.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:16.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:16.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:16.246 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:16.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:16.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:16.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:17.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:16 smithi006 ceph-mon[97840]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:16 smithi006 ceph-mon[97840]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:16 smithi087 ceph-mon[131182]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:16 smithi087 ceph-mon[131182]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:16 smithi186 ceph-mon[131487]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:16 smithi186 ceph-mon[131487]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:18.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/3365401197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:17 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:17 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]': finished 2023-11-17T00:03:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:17 smithi006 ceph-mon[97840]: osdmap e48: 8 total, 7 up, 8 in 2023-11-17T00:03:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:17 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/3365401197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:17 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:17 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]': finished 2023-11-17T00:03:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:17 smithi087 ceph-mon[131182]: osdmap e48: 8 total, 7 up, 8 in 2023-11-17T00:03:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:17 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/3365401197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:17 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]: dispatch 2023-11-17T00:03:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:17 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9282e04d-a458-456a-ac74-a942aa06cdc5"}]': finished 2023-11-17T00:03:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:17 smithi186 ceph-mon[131487]: osdmap e48: 8 total, 7 up, 8 in 2023-11-17T00:03:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:17 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:19.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:18 smithi006 ceph-mon[97840]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:19.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/925439003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:18 smithi087 ceph-mon[131182]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/925439003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:19.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:18 smithi186 ceph-mon[131487]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:19.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/925439003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:20 smithi006 ceph-mon[97840]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:20 smithi087 ceph-mon[131182]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:21.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:20 smithi186 ceph-mon[131487]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:22 smithi006 ceph-mon[97840]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:22 smithi087 ceph-mon[131182]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:22 smithi186 ceph-mon[131487]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:24.598 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:24 smithi087 ceph-mon[131182]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:24.598 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:24 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-17T00:03:24.599 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:24 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:24.599 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:24 smithi087 ceph-mon[131182]: Deploying daemon osd.7 on smithi087 2023-11-17T00:03:24.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:24 smithi186 ceph-mon[131487]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:24.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:24 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-17T00:03:24.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:24 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:24.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:24 smithi186 ceph-mon[131487]: Deploying daemon osd.7 on smithi087 2023-11-17T00:03:24.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:24 smithi006 ceph-mon[97840]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:24.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:24 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-17T00:03:24.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:24 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:24.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:24 smithi006 ceph-mon[97840]: Deploying daemon osd.7 on smithi087 2023-11-17T00:03:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:26 smithi087 ceph-mon[131182]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:26 smithi186 ceph-mon[131487]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:26 smithi006 ceph-mon[97840]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:28 smithi087 ceph-mon[131182]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:28 smithi186 ceph-mon[131487]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:28 smithi006 ceph-mon[97840]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:30.877 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.875+0000 7fd552ffd700 1 -- 172.21.15.87:0/3429187651 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd560060f30 con 0x7fd54c0a1150 2023-11-17T00:03:30.877 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 7 on host 'smithi087' 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 msgr2=0x7fd54c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 0x7fd54c0a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005950 tx=0x7fd55c0058e0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 msgr2=0x7fd560194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c920 tx=0x7fd55400cce0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 shutdown_connections 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd54c0a1150 0x7fd54c0a3600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd560100860 0x7fd56019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5600ff430 0x7fd560194fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.879 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 --2- 172.21.15.87:0/3429187651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5600ffde0 0x7fd560195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:30.880 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 >> 172.21.15.87:0/3429187651 conn(0x7fd5600f9f70 msgr2=0x7fd5600fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:03:30.880 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.877+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 shutdown_connections 2023-11-17T00:03:30.880 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:03:30.878+0000 7fd567da7700 1 -- 172.21.15.87:0/3429187651 wait complete. 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:31.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:31.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:31.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:31.266 DEBUG:teuthology.orchestra.run.smithi087:osd.7> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.7.service 2023-11-17T00:03:31.269 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2023-11-17T00:03:31.270 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-17T00:03:31.301 INFO:journalctl@ceph.osd.7.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:03:31.921 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:03:32.011 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.011 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:03:32.011 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.012 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.012 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.012 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:31 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:32.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:03:32.133 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.133 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.134 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.134 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:31 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:32.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:03:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:31 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:32.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:03:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:03:32.158+0000 7f3bda66f640 -1 osd.7 0 log_to_monitors true 2023-11-17T00:03:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:32 smithi006 ceph-mon[97840]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:32 smithi006 ceph-mon[97840]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:32 smithi006 ceph-mon[97840]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:32 smithi087 ceph-mon[131182]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:33.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:32 smithi087 ceph-mon[131182]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:32 smithi087 ceph-mon[131182]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:32 smithi186 ceph-mon[131487]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:32 smithi186 ceph-mon[131487]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:32 smithi186 ceph-mon[131487]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01273 s, 824 MB/s 2023-11-17T00:03:33.523 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-11-17T00:03:33.962 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: osdmap e49: 8 total, 7 up, 8 in 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:34.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:34.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:33 smithi006 ceph-mon[97840]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:03:34.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:03:33.882+0000 7f3bcfce6700 -1 osd.7 0 waiting for initial osdmap 2023-11-17T00:03:34.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:03:33.892+0000 7f3bc9ad7700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: osdmap e49: 8 total, 7 up, 8 in 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:34.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:34.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.364 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:33 smithi087 ceph-mon[131182]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:03:34.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: osdmap e49: 8 total, 7 up, 8 in 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:34.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:34.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:33 smithi186 ceph-mon[131487]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:03:34.600 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi087 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 5722M 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: osdmap e50: 8 total, 7 up, 8 in 2023-11-17T00:03:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:35.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi087 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 5722M 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: osdmap e50: 8 total, 7 up, 8 in 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi087 2023-11-17T00:03:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 5722M 2023-11-17T00:03:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:03:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-17T00:03:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: osdmap e50: 8 total, 7 up, 8 in 2023-11-17T00:03:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 -- 172.21.15.186:0/1535527517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 msgr2=0x7fdbf0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 --2- 172.21.15.186:0/1535527517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0100720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009990 tx=0x7fdbe0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 -- 172.21.15.186:0/1535527517 shutdown_connections 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 --2- 172.21.15.186:0/1535527517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 0x7fdbf0108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 --2- 172.21.15.186:0/1535527517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0100720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 --2- 172.21.15.186:0/1535527517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 0x7fdbf0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 -- 172.21.15.186:0/1535527517 >> 172.21.15.186:0/1535527517 conn(0x7fdbf00fb960 msgr2=0x7fdbf00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:03:36.018 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 -- 172.21.15.186:0/1535527517 shutdown_connections 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.014+0000 7fdbf749a700 1 -- 172.21.15.186:0/1535527517 wait complete. 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 Processor -- start 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 -- start start 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 0x7fdbf01966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 0x7fdbf019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019dcf0 con 0x7fdbf0101ea0 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019de30 con 0x7fdbf0101510 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf749a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf019dfc0 con 0x7fdbf0100310 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf5236700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf5236700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55798/0 (socket says 172.21.15.186:55798) 2023-11-17T00:03:36.019 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.015+0000 7fdbf5236700 1 -- 172.21.15.186:0/3529711186 learned_addr learned my addr 172.21.15.186:0/3529711186 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf4a35700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 0x7fdbf01966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 -- 172.21.15.186:0/3529711186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 msgr2=0x7fdbf01966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 0x7fdbf01966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 -- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 msgr2=0x7fdbf019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5a37700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 0x7fdbf019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 0x7fdbf019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 -- 172.21.15.186:0/3529711186 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00095f0 con 0x7fdbf0100310 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf5236700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b5c0 tx=0x7fdbe000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe001d070 con 0x7fdbf0100310 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe0022470 con 0x7fdbf0100310 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf010bce0 con 0x7fdbf0100310 2023-11-17T00:03:36.020 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0017630 con 0x7fdbf0100310 2023-11-17T00:03:36.021 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.016+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf010bb50 con 0x7fdbf0100310 2023-11-17T00:03:36.021 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.017+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf0107960 con 0x7fdbf0100310 2023-11-17T00:03:36.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.018+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdbe00228e0 con 0x7fdbf0100310 2023-11-17T00:03:36.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.019+0000 7fdbe67fc700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 0x7fdbdc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:03:36.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.019+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fdbe00cd360 con 0x7fdbf0100310 2023-11-17T00:03:36.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.019+0000 7fdbf4a35700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 0x7fdbdc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:03:36.022 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.020+0000 7fdbf4a35700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 0x7fdbdc0a3680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbec005950 tx=0x7fdbec00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:03:36.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.022+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbe0097680 con 0x7fdbf0100310 2023-11-17T00:03:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:35 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:03:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:35 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:03:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:35 smithi186 ceph-mon[131487]: osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] boot 2023-11-17T00:03:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:35 smithi186 ceph-mon[131487]: osdmap e51: 8 total, 8 up, 8 in 2023-11-17T00:03:36.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:35 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:36.217 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:36.216+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf0061c50 con 0x7fdbdc0a11d0 2023-11-17T00:03:36.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:35 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:03:36.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:35 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:03:36.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:35 smithi006 ceph-mon[97840]: osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] boot 2023-11-17T00:03:36.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:35 smithi006 ceph-mon[97840]: osdmap e51: 8 total, 8 up, 8 in 2023-11-17T00:03:36.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:35 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:36.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:35 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:03:36.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:35 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:03:36.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:35 smithi087 ceph-mon[131182]: osd.7 [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] boot 2023-11-17T00:03:36.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:35 smithi087 ceph-mon[131182]: osdmap e51: 8 total, 8 up, 8 in 2023-11-17T00:03:36.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:35 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:03:37.139 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:37.139 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: osdmap e52: 8 total, 8 up, 8 in 2023-11-17T00:03:37.140 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:37.140 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:37.140 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:37.140 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:36 smithi186 ceph-mon[131487]: osdmap e53: 8 total, 8 up, 8 in 2023-11-17T00:03:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: osdmap e52: 8 total, 8 up, 8 in 2023-11-17T00:03:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:36 smithi006 ceph-mon[97840]: osdmap e53: 8 total, 8 up, 8 in 2023-11-17T00:03:37.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:37.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: osdmap e52: 8 total, 8 up, 8 in 2023-11-17T00:03:37.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:03:37.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:03:37.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:37.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:36 smithi087 ceph-mon[131182]: osdmap e53: 8 total, 8 up, 8 in 2023-11-17T00:03:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:37 smithi186 ceph-mon[131487]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:38.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:37 smithi006 ceph-mon[97840]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:37 smithi087 ceph-mon[131182]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:03:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: osdmap e54: 8 total, 8 up, 8 in 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/3061491638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]': finished 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: osdmap e55: 9 total, 8 up, 9 in 2023-11-17T00:03:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:03:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: osdmap e54: 8 total, 8 up, 8 in 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/3061491638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]': finished 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: osdmap e55: 9 total, 8 up, 9 in 2023-11-17T00:03:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:03:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: osdmap e54: 8 total, 8 up, 8 in 2023-11-17T00:03:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/3061491638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]: dispatch 2023-11-17T00:03:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02359124-89e5-4f4a-a547-b5fe9e03d316"}]': finished 2023-11-17T00:03:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: osdmap e55: 9 total, 8 up, 9 in 2023-11-17T00:03:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:03:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:39 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/3443242657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:40.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:39 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/3443242657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:40.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:39 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/3443242657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:03:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:40 smithi006 ceph-mon[97840]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:40 smithi087 ceph-mon[131182]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:40 smithi186 ceph-mon[131487]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:43.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:42 smithi006 ceph-mon[97840]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:42 smithi087 ceph-mon[131182]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:42 smithi186 ceph-mon[131487]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:44.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:44 smithi087 ceph-mon[131182]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:44.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:44 smithi186 ceph-mon[131487]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:44.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:44 smithi006 ceph-mon[97840]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:45.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-17T00:03:45.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:45.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-17T00:03:45.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:45.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-17T00:03:45.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:46.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:46 smithi087 ceph-mon[131182]: Deploying daemon osd.8 on smithi186 2023-11-17T00:03:46.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:46 smithi087 ceph-mon[131182]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:46.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:46 smithi186 ceph-mon[131487]: Deploying daemon osd.8 on smithi186 2023-11-17T00:03:46.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:46 smithi186 ceph-mon[131487]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:46 smithi006 ceph-mon[97840]: Deploying daemon osd.8 on smithi186 2023-11-17T00:03:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:46 smithi006 ceph-mon[97840]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:48.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:48 smithi186 ceph-mon[131487]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:48 smithi087 ceph-mon[131182]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:48 smithi006 ceph-mon[97840]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:50.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:50 smithi186 ceph-mon[131487]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:50.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:50.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:50.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:51.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:50 smithi087 ceph-mon[131182]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:51.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:50 smithi006 ceph-mon[97840]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:51.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:52 smithi087 ceph-mon[131182]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:52 smithi186 ceph-mon[131487]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:53.188 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.186+0000 7fdbe67fc700 1 -- 172.21.15.186:0/3529711186 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdbf0061c50 con 0x7fdbdc0a11d0 2023-11-17T00:03:53.188 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2023-11-17T00:03:53.190 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.188+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 msgr2=0x7fdbdc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.188+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 0x7fdbdc0a3680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdbec005950 tx=0x7fdbec00b410 comp rx=0 tx=0).stop 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.188+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 msgr2=0x7fdbf0196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.188+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b5c0 tx=0x7fdbe000f690 comp rx=0 tx=0).stop 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 shutdown_connections 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbdc0a11d0 0x7fdbdc0a3680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf0101ea0 0x7fdbf019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:53.191 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf0100310 0x7fdbf0196160 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:53.192 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 --2- 172.21.15.186:0/3529711186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf0101510 0x7fdbf01966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:03:53.192 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 >> 172.21.15.186:0/3529711186 conn(0x7fdbf00fb960 msgr2=0x7fdbf0104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:03:53.192 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 shutdown_connections 2023-11-17T00:03:53.192 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:03:53.189+0000 7fdbf749a700 1 -- 172.21.15.186:0/3529711186 wait complete. 2023-11-17T00:03:53.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:52 smithi006 ceph-mon[97840]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:03:53.596 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.8.service 2023-11-17T00:03:53.598 INFO:tasks.cephadm:Deploying osd.9 on smithi006 with /dev/vg_nvme/lv_1... 2023-11-17T00:03:53.599 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-17T00:03:53.628 INFO:journalctl@ceph.osd.8.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:03:53.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:03:53.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:53.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:53 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:53.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:53 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:03:54.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:53 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:03:54.365 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:03:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:03:54.158+0000 7f421a27b640 -1 osd.8 0 log_to_monitors true 2023-11-17T00:04:04.945 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:04:05.322 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:54 smithi186 ceph-mon[131487]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:05.832 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:54 smithi186 ceph-mon[131487]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:05.941 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:54 smithi186 ceph-mon[131487]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:06.356 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-mon[131487]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-17T00:04:06.489 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-mon[131487]: osdmap e56: 9 total, 8 up, 9 in 2023-11-17T00:04:06.563 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:06.819 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-mon[131487]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:06.819 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-mon[131487]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:06.820 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:54 smithi006 ceph-mon[97840]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:06.882 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:54 smithi006 ceph-mon[97840]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:06.946 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:54 smithi006 ceph-mon[97840]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:07.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:55 smithi006 ceph-mon[97840]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-17T00:04:07.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:55 smithi006 ceph-mon[97840]: osdmap e56: 9 total, 8 up, 9 in 2023-11-17T00:04:07.283 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:07.349 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:55 smithi006 ceph-mon[97840]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:07.482 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:55 smithi006 ceph-mon[97840]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:07.483 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:03:55.856+0000 7f420f8f2700 -1 osd.8 0 waiting for initial osdmap 2023-11-17T00:04:07.548 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:03:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:03:55.870+0000 7f4207ee0700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:04:07.548 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-17T00:04:07.609 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-17T00:04:07.675 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2023-11-17T00:04:07.750 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2023-11-17T00:04:07.750 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.046515 s, 225 MB/s 2023-11-17T00:04:07.750 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2023-11-17T00:04:07.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:07.776 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:04:07.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: osdmap e57: 9 total, 8 up, 9 in 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:08.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:08.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.010 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:54 smithi087 ceph-mon[131182]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:08.011 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:54 smithi087 ceph-mon[131182]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:08.011 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:54 smithi087 ceph-mon[131182]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-17T00:04:08.012 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:55 smithi087 ceph-mon[131182]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:55 smithi087 ceph-mon[131182]: osdmap e56: 9 total, 8 up, 9 in 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:55 smithi087 ceph-mon[131182]: from='osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:55 smithi087 ceph-mon[131182]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: osdmap e57: 9 total, 8 up, 9 in 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.193 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:04:08.194 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi186 2023-11-17T00:04:08.195 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 5722M 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] boot 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: osdmap e58: 9 total, 9 up, 9 in 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:57 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:58 smithi087 ceph-mon[131182]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:58 smithi087 ceph-mon[131182]: osdmap e59: 9 total, 9 up, 9 in 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:03:59 smithi087 ceph-mon[131182]: osdmap e60: 9 total, 9 up, 9 in 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:00 smithi087 ceph-mon[131182]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:00 smithi087 ceph-mon[131182]: osdmap e61: 9 total, 9 up, 9 in 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:01 smithi087 ceph-mon[131182]: osdmap e62: 9 total, 9 up, 9 in 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:02 smithi087 ceph-mon[131182]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:04 smithi087 ceph-mon[131182]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:06 smithi087 ceph-mon[131182]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi006:vg_nvme/lv_1 2023-11-17T00:04:08.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-17T00:04:08.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:04:08.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: osdmap e57: 9 total, 8 up, 9 in 2023-11-17T00:04:08.418 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi186 2023-11-17T00:04:08.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 5722M 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] boot 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: osdmap e58: 9 total, 9 up, 9 in 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:57 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:58 smithi186 ceph-mon[131487]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:58 smithi186 ceph-mon[131487]: osdmap e59: 9 total, 9 up, 9 in 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:03:59 smithi186 ceph-mon[131487]: osdmap e60: 9 total, 9 up, 9 in 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:00 smithi186 ceph-mon[131487]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:00 smithi186 ceph-mon[131487]: osdmap e61: 9 total, 9 up, 9 in 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:01 smithi186 ceph-mon[131487]: osdmap e62: 9 total, 9 up, 9 in 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:02 smithi186 ceph-mon[131487]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.420 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:04 smithi186 ceph-mon[131487]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.421 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:06 smithi186 ceph-mon[131487]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:04:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:04:08.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi186 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 5722M 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: osd.8 [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] boot 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: osdmap e58: 9 total, 9 up, 9 in 2023-11-17T00:04:08.422 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:57 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:04:08.775 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:58 smithi006 ceph-mon[97840]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.775 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:58 smithi006 ceph-mon[97840]: osdmap e59: 9 total, 9 up, 9 in 2023-11-17T00:04:08.784 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:03:59 smithi006 ceph-mon[97840]: osdmap e60: 9 total, 9 up, 9 in 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:00 smithi006 ceph-mon[97840]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:00 smithi006 ceph-mon[97840]: osdmap e61: 9 total, 9 up, 9 in 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:01 smithi006 ceph-mon[97840]: osdmap e62: 9 total, 9 up, 9 in 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:02 smithi006 ceph-mon[97840]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:04 smithi006 ceph-mon[97840]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-11-17T00:04:08.785 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:06 smithi006 ceph-mon[97840]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-11-17T00:04:09.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:08 smithi087 ceph-mon[131182]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:04:09.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:08 smithi186 ceph-mon[131487]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:04:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:08 smithi006 ceph-mon[97840]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:04:09.656 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:04:11.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:10 smithi087 ceph-mon[131182]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-17T00:04:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:10 smithi186 ceph-mon[131487]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-17T00:04:11.172 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:10 smithi006 ceph-mon[97840]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 -- 172.21.15.6:0/1335589959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e481081c0 msgr2=0x7f4e4810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 --2- 172.21.15.6:0/1335589959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e481081c0 0x7f4e4810a600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000b0a0 tx=0x7f4e4000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 -- 172.21.15.6:0/1335589959 shutdown_connections 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 --2- 172.21.15.6:0/1335589959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e481081c0 0x7f4e4810a600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 --2- 172.21.15.6:0/1335589959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e48074b20 0x7f4e48072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 --2- 172.21.15.6:0/1335589959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 0x7f4e480739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 -- 172.21.15.6:0/1335589959 >> 172.21.15.6:0/1335589959 conn(0x7f4e480fb900 msgr2=0x7f4e480fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:04:11.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 -- 172.21.15.6:0/1335589959 shutdown_connections 2023-11-17T00:04:11.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.454+0000 7f4e4da06700 1 -- 172.21.15.6:0/1335589959 wait complete. 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.455+0000 7f4e4da06700 1 Processor -- start 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.455+0000 7f4e4da06700 1 -- start start 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 0x7f4e48198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481081c0 0x7f4e4819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4819fec0 con 0x7f4e48074b20 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481a0000 con 0x7f4e481081c0 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e4da06700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481a0190 con 0x7f4e48073550 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 0x7f4e48198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:11.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54450/0 (socket says 172.21.15.6:54450) 2023-11-17T00:04:11.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e467fc700 1 -- 172.21.15.6:0/3725010100 learned_addr learned my addr 172.21.15.6:0/3725010100 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:04:11.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.456+0000 7f4e477fe700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481081c0 0x7f4e4819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:11.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 -- 172.21.15.6:0/3725010100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 msgr2=0x7f4e48198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 0x7f4e48198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 -- 172.21.15.6:0/3725010100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481081c0 msgr2=0x7f4e4819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481081c0 0x7f4e4819f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 -- 172.21.15.6:0/3725010100 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009d00 con 0x7f4e48074b20 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e467fc700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e9d0 tx=0x7f4e3c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c009790 con 0x7f4e48074b20 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3c00ee70 con 0x7f4e48074b20 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c010ea0 con 0x7f4e48074b20 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4810de90 con 0x7f4e48074b20 2023-11-17T00:04:11.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.457+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4810dd00 con 0x7f4e48074b20 2023-11-17T00:04:11.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.458+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48106870 con 0x7f4e48074b20 2023-11-17T00:04:11.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.461+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4e3c00cbd0 con 0x7f4e48074b20 2023-11-17T00:04:11.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.462+0000 7f4e4ca04700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 0x7f4e300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:11.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.462+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f4e3c014070 con 0x7f4e48074b20 2023-11-17T00:04:11.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.462+0000 7f4e46ffd700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 0x7f4e300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:11.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.463+0000 7f4e46ffd700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 0x7f4e300a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009c60 tx=0x7f4e38009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:04:11.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.464+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4e3c096640 con 0x7f4e48074b20 2023-11-17T00:04:11.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:11.681+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e48061c90 con 0x7f4e300a11d0 2023-11-17T00:04:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:12.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:11 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:12.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:12.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:12.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:11 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:12.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:12.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:11 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:13.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:12 smithi087 ceph-mon[131182]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:12 smithi087 ceph-mon[131182]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:12 smithi186 ceph-mon[131487]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:12 smithi186 ceph-mon[131487]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:13.127 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:12 smithi006 ceph-mon[97840]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:13.127 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:12 smithi006 ceph-mon[97840]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:14 smithi006 ceph-mon[97840]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:14.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:14 smithi087 ceph-mon[131182]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:14.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:14 smithi186 ceph-mon[131487]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:15.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]: dispatch 2023-11-17T00:04:15.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]': finished 2023-11-17T00:04:15.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:15 smithi087 ceph-mon[131182]: osdmap e63: 10 total, 9 up, 10 in 2023-11-17T00:04:15.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:15 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:15.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]: dispatch 2023-11-17T00:04:15.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]': finished 2023-11-17T00:04:15.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:15 smithi186 ceph-mon[131487]: osdmap e63: 10 total, 9 up, 10 in 2023-11-17T00:04:15.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:15 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:15.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]: dispatch 2023-11-17T00:04:15.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3917333808' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b7d811-9e00-427c-a0b7-e300fd19a9df"}]': finished 2023-11-17T00:04:15.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:15 smithi006 ceph-mon[97840]: osdmap e63: 10 total, 9 up, 10 in 2023-11-17T00:04:15.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:15 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:16.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:16 smithi087 ceph-mon[131182]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:16.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1052463787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:16 smithi186 ceph-mon[131487]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:16.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1052463787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:16.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:16 smithi006 ceph-mon[97840]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:16.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1052463787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:18 smithi087 ceph-mon[131182]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:18 smithi186 ceph-mon[131487]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:19.133 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:18 smithi006 ceph-mon[97840]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:20 smithi087 ceph-mon[131182]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:20 smithi186 ceph-mon[131487]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:21.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:20 smithi006 ceph-mon[97840]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:22.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:22 smithi006 ceph-mon[97840]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:22 smithi087 ceph-mon[131182]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:22 smithi186 ceph-mon[131487]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:24.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-17T00:04:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:23 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-17T00:04:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:23 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:24.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-17T00:04:24.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:23 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:24.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:24 smithi006 ceph-mon[97840]: Deploying daemon osd.9 on smithi006 2023-11-17T00:04:24.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:24 smithi006 ceph-mon[97840]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:25.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:24 smithi087 ceph-mon[131182]: Deploying daemon osd.9 on smithi006 2023-11-17T00:04:25.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:24 smithi087 ceph-mon[131182]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:24 smithi186 ceph-mon[131487]: Deploying daemon osd.9 on smithi006 2023-11-17T00:04:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:24 smithi186 ceph-mon[131487]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:26.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:26 smithi006 ceph-mon[97840]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:26 smithi087 ceph-mon[131182]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:26 smithi186 ceph-mon[131487]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:29.044 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:28 smithi006 ceph-mon[97840]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:28 smithi087 ceph-mon[131182]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:28 smithi186 ceph-mon[131487]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:30 smithi006 ceph-mon[97840]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:31.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:30 smithi087 ceph-mon[131182]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:30 smithi186 ceph-mon[131487]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:32 smithi087 ceph-mon[131182]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:33.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:32 smithi006 ceph-mon[97840]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:32 smithi186 ceph-mon[131487]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:34.361 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 9 on host 'smithi006' 2023-11-17T00:04:34.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.359+0000 7f4e4ca04700 1 -- 172.21.15.6:0/3725010100 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e48061c90 con 0x7f4e300a11d0 2023-11-17T00:04:34.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 msgr2=0x7f4e300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:34.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 0x7f4e300a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009c60 tx=0x7f4e38009260 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 msgr2=0x7f4e48198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e9d0 tx=0x7f4e3c00ece0 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 shutdown_connections 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4e300a11d0 0x7f4e300a3680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481081c0 0x7f4e4819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e48073550 0x7f4e48198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 --2- 172.21.15.6:0/3725010100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e48074b20 0x7f4e48198870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 >> 172.21.15.6:0/3725010100 conn(0x7f4e480fb900 msgr2=0x7f4e480fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 shutdown_connections 2023-11-17T00:04:34.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:04:34.362+0000 7f4e4da06700 1 -- 172.21.15.6:0/3725010100 wait complete. 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.841 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.842 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:34.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:34.871 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.871 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.871 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:04:34.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:34.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:34.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-17T00:04:34.874 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:34.886 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:34.898 DEBUG:teuthology.orchestra.run.smithi006:osd.9> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.9.service 2023-11-17T00:04:34.905 INFO:tasks.cephadm:Deploying osd.10 on smithi087 with /dev/vg_nvme/lv_1... 2023-11-17T00:04:34.905 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-17T00:04:34.978 INFO:journalctl@ceph.osd.9.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:04:35.556 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:04:35.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:35 smithi087 ceph-mon[131182]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-17T00:04:35.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:35 smithi087 ceph-mon[131182]: osdmap e64: 10 total, 9 up, 10 in 2023-11-17T00:04:35.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:35 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:35.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:35 smithi087 ceph-mon[131182]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.799 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:35 smithi087 ceph-mon[131182]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:35 smithi186 ceph-mon[131487]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-17T00:04:35.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:35 smithi186 ceph-mon[131487]: osdmap e64: 10 total, 9 up, 10 in 2023-11-17T00:04:35.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:35 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:35.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:35 smithi186 ceph-mon[131487]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:35 smithi186 ceph-mon[131487]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-mon[97840]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-17T00:04:35.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-mon[97840]: osdmap e64: 10 total, 9 up, 10 in 2023-11-17T00:04:35.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:35.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-mon[97840]: from='osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.879 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-mon[97840]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2023-11-17T00:04:35.879 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:04:35.547+0000 7fe2ed386700 -1 osd.9 0 waiting for initial osdmap 2023-11-17T00:04:35.879 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:04:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:04:35.567+0000 7fe2e6175700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:04:36.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:36 smithi087 ceph-mon[131182]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:04:36.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:36 smithi087 ceph-mon[131182]: osdmap e65: 10 total, 9 up, 10 in 2023-11-17T00:04:36.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:36 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:36 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:36 smithi087 ceph-mon[131182]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:36 smithi186 ceph-mon[131487]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:04:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:36 smithi186 ceph-mon[131487]: osdmap e65: 10 total, 9 up, 10 in 2023-11-17T00:04:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:36 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:36 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:36 smithi186 ceph-mon[131487]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:36.930 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-17T00:04:36.931 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-17T00:04:36.931 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2023-11-17T00:04:36.931 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2023-11-17T00:04:36.931 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112789 s, 930 MB/s 2023-11-17T00:04:36.931 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2023-11-17T00:04:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:36 smithi006 ceph-mon[97840]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2023-11-17T00:04:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:36 smithi006 ceph-mon[97840]: osdmap e65: 10 total, 9 up, 10 in 2023-11-17T00:04:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:36 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:36 smithi006 ceph-mon[97840]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-17T00:04:37.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi087:vg_nvme/lv_1 2023-11-17T00:04:37.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:37 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:04:37.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:37 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:04:37.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:37 smithi087 ceph-mon[131182]: osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] boot 2023-11-17T00:04:37.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:37 smithi087 ceph-mon[131182]: osdmap e66: 10 total, 10 up, 10 in 2023-11-17T00:04:37.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:37.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:37 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:04:37.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:37 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:04:37.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:37 smithi006 ceph-mon[97840]: osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] boot 2023-11-17T00:04:37.706 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:37 smithi006 ceph-mon[97840]: osdmap e66: 10 total, 10 up, 10 in 2023-11-17T00:04:37.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:37.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:37 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:04:37.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:37 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:04:37.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:37 smithi186 ceph-mon[131487]: osd.9 [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] boot 2023-11-17T00:04:37.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:37 smithi186 ceph-mon[131487]: osdmap e66: 10 total, 10 up, 10 in 2023-11-17T00:04:37.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:04:37.954 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: osdmap e67: 10 total, 10 up, 10 in 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi006 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 4319M 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:04:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:38 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: osdmap e67: 10 total, 10 up, 10 in 2023-11-17T00:04:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi006 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 4319M 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:38.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:04:38.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:38 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: osdmap e67: 10 total, 10 up, 10 in 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi006 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 4319M 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:04:38.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:38 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 -- 172.21.15.87:0/1043190771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 msgr2=0x7f5e50105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 --2- 172.21.15.87:0/1043190771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50105ad0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009990 tx=0x7f5e38009ca0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 -- 172.21.15.87:0/1043190771 shutdown_connections 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 --2- 172.21.15.87:0/1043190771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 --2- 172.21.15.87:0/1043190771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 0x7f5e50108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.105 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 --2- 172.21.15.87:0/1043190771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50105ad0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.106 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 -- 172.21.15.87:0/1043190771 >> 172.21.15.87:0/1043190771 conn(0x7f5e500fa7f0 msgr2=0x7f5e500fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:04:39.106 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 -- 172.21.15.87:0/1043190771 shutdown_connections 2023-11-17T00:04:39.106 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.103+0000 7f5e5623f700 1 -- 172.21.15.87:0/1043190771 wait complete. 2023-11-17T00:04:39.106 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 Processor -- start 2023-11-17T00:04:39.106 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 -- start start 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e50196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 0x7f5e5019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019dc90 con 0x7f5e50100dd0 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019ddd0 con 0x7f5e501036f0 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e5623f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019df60 con 0x7f5e50106010 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e4effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:39.108 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e4effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35718/0 (socket says 172.21.15.87:35718) 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.104+0000 7f5e4effd700 1 -- 172.21.15.87:0/115675538 learned_addr learned my addr 172.21.15.87:0/115675538 (peer_addr_for_me v2:172.21.15.87:0/0) 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4f7fe700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e50196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4ffff700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 0x7f5e5019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 -- 172.21.15.87:0/115675538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 msgr2=0x7f5e5019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 0x7f5e5019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 -- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 msgr2=0x7f5e50196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e50196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 -- 172.21.15.87:0/115675538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380095f0 con 0x7f5e501036f0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4f7fe700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e50196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4effd700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e38006010 tx=0x7f5e3800f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3801d070 con 0x7f5e501036f0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5010bcd0 con 0x7f5e501036f0 2023-11-17T00:04:39.109 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3800fc60 con 0x7f5e501036f0 2023-11-17T00:04:39.110 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380176e0 con 0x7f5e501036f0 2023-11-17T00:04:39.110 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.105+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5010bb40 con 0x7f5e501036f0 2023-11-17T00:04:39.110 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.106+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5019e500 con 0x7f5e501036f0 2023-11-17T00:04:39.110 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.107+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5e380229c0 con 0x7f5e501036f0 2023-11-17T00:04:39.111 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.108+0000 7f5e4cff9700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 0x7f5e3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:04:39.111 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.108+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5e380cdaa0 con 0x7f5e501036f0 2023-11-17T00:04:39.111 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.109+0000 7f5e4f7fe700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 0x7f5e3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:04:39.111 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.109+0000 7f5e4f7fe700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 0x7f5e3c0a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005fd0 tx=0x7f5e40005dc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:04:39.113 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.111+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e38097a60 con 0x7f5e501036f0 2023-11-17T00:04:39.306 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:39.305+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e50102920 con 0x7f5e3c0a1150 2023-11-17T00:04:40.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:39 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:40.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:39 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:40.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:39 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:40.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:39 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:40.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:39 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:40.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:39 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:04:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:04:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:39 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:40.901 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:40 smithi087 ceph-mon[131182]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:40.901 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:40 smithi087 ceph-mon[131182]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:40 smithi186 ceph-mon[131487]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:40 smithi186 ceph-mon[131487]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:40 smithi006 ceph-mon[97840]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:04:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:40 smithi006 ceph-mon[97840]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:42.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:41 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/3661814846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:41 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:41 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]': finished 2023-11-17T00:04:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:41 smithi087 ceph-mon[131182]: osdmap e68: 11 total, 10 up, 11 in 2023-11-17T00:04:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:41 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:42.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:41 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/3661814846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:41 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:41 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]': finished 2023-11-17T00:04:42.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:41 smithi186 ceph-mon[131487]: osdmap e68: 11 total, 10 up, 11 in 2023-11-17T00:04:42.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:41 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/3661814846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:41 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]: dispatch 2023-11-17T00:04:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:41 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5acfeda0-5332-4020-bcc7-2173606e96ad"}]': finished 2023-11-17T00:04:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:41 smithi006 ceph-mon[97840]: osdmap e68: 11 total, 10 up, 11 in 2023-11-17T00:04:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:41 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:42 smithi087 ceph-mon[131182]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.87:0/1973361293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:42 smithi186 ceph-mon[131487]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.87:0/1973361293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:43.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:42 smithi006 ceph-mon[97840]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.87:0/1973361293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:04:44.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:44 smithi087 ceph-mon[131182]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:44.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:44 smithi186 ceph-mon[131487]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:44.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:44 smithi006 ceph-mon[97840]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:46.867 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:46 smithi087 ceph-mon[131182]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:46 smithi186 ceph-mon[131487]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:47.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:46 smithi006 ceph-mon[97840]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:49.001 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:48 smithi087 ceph-mon[131182]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:49.001 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:48 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-17T00:04:49.001 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:48 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:48 smithi186 ceph-mon[131487]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:48 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-17T00:04:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:48 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:48 smithi006 ceph-mon[97840]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:48 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-17T00:04:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:48 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:50.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:49 smithi087 ceph-mon[131182]: Deploying daemon osd.10 on smithi087 2023-11-17T00:04:50.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:49 smithi186 ceph-mon[131487]: Deploying daemon osd.10 on smithi087 2023-11-17T00:04:50.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:49 smithi006 ceph-mon[97840]: Deploying daemon osd.10 on smithi087 2023-11-17T00:04:50.787 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:50 smithi087 ceph-mon[131182]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:50 smithi186 ceph-mon[131487]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:51.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:50 smithi006 ceph-mon[97840]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:53.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:52 smithi087 ceph-mon[131182]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:52 smithi186 ceph-mon[131487]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:53.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:52 smithi006 ceph-mon[97840]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:54.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:54 smithi087 ceph-mon[131182]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:54.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:54 smithi186 ceph-mon[131487]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:54.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:54 smithi006 ceph-mon[97840]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:55.219 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.218+0000 7f5e4cff9700 1 -- 172.21.15.87:0/115675538 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f5e50102920 con 0x7f5e3c0a1150 2023-11-17T00:04:55.219 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 10 on host 'smithi087' 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 msgr2=0x7f5e3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 0x7f5e3c0a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005fd0 tx=0x7f5e40005dc0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 msgr2=0x7f5e50196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e38006010 tx=0x7f5e3800f7b0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 shutdown_connections 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e3c0a1150 0x7f5e3c0a3600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.222 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e50100dd0 0x7f5e50196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.220+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e50106010 0x7f5e5019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.221+0000 7f5e5623f700 1 --2- 172.21.15.87:0/115675538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e501036f0 0x7f5e50196640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:04:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.221+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 >> 172.21.15.87:0/115675538 conn(0x7f5e500fa7f0 msgr2=0x7f5e50102210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:04:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.221+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 shutdown_connections 2023-11-17T00:04:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2023-11-17T00:04:55.221+0000 7f5e5623f700 1 -- 172.21.15.87:0/115675538 wait complete. 2023-11-17T00:04:55.542 DEBUG:teuthology.orchestra.run.smithi087:osd.10> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.10.service 2023-11-17T00:04:55.544 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2023-11-17T00:04:55.545 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-17T00:04:55.573 INFO:journalctl@ceph.osd.10.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:55 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:55 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:55.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:55 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:04:56.202 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:04:56.351 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:04:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:04:56.163+0000 7fa64b525640 -1 osd.10 0 log_to_monitors true 2023-11-17T00:04:56.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:56 smithi186 ceph-mon[131487]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:56.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:56 smithi186 ceph-mon[131487]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:56.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:56 smithi186 ceph-mon[131487]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:56.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:56 smithi087 ceph-mon[131182]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:56.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:56 smithi087 ceph-mon[131182]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:56.913 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:56 smithi087 ceph-mon[131182]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:56.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:56 smithi006 ceph-mon[97840]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:56.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:56 smithi006 ceph-mon[97840]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:56.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:56 smithi006 ceph-mon[97840]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-17T00:04:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-17T00:04:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-17T00:04:57.667 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-11-17T00:04:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-11-17T00:04:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135246 s, 775 MB/s 2023-11-17T00:04:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-11-17T00:04:57.743 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:04:57.649+0000 7fa640b9c700 -1 osd.10 0 waiting for initial osdmap 2023-11-17T00:04:57.744 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:04:57.663+0000 7fa63918a700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:04:57.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-mon[131182]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-17T00:04:57.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-mon[131182]: osdmap e69: 11 total, 10 up, 11 in 2023-11-17T00:04:57.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:57.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-mon[131182]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:57.744 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:57 smithi087 ceph-mon[131182]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:57.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:57 smithi006 ceph-mon[97840]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-17T00:04:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:57 smithi006 ceph-mon[97840]: osdmap e69: 11 total, 10 up, 11 in 2023-11-17T00:04:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:57 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:57 smithi006 ceph-mon[97840]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:57 smithi006 ceph-mon[97840]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:58.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:57 smithi186 ceph-mon[131487]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-17T00:04:58.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:57 smithi186 ceph-mon[131487]: osdmap e69: 11 total, 10 up, 11 in 2023-11-17T00:04:58.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:57 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:57 smithi186 ceph-mon[131487]: from='osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:58.055 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:57 smithi186 ceph-mon[131487]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2023-11-17T00:04:58.057 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2023-11-17T00:04:58.697 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: osdmap e70: 11 total, 10 up, 11 in 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi087 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.947 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 4292M 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.948 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:04:58.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:04:58.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] boot 2023-11-17T00:04:58.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: osdmap e71: 11 total, 11 up, 11 in 2023-11-17T00:04:58.949 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:04:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: osdmap e70: 11 total, 10 up, 11 in 2023-11-17T00:04:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi087 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 4292M 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:58.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:04:58.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:04:58.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] boot 2023-11-17T00:04:58.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: osdmap e71: 11 total, 11 up, 11 in 2023-11-17T00:04:58.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: osdmap e70: 11 total, 10 up, 11 in 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi087 2023-11-17T00:04:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 4292M 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:04:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: osd.10 [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] boot 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: osdmap e71: 11 total, 11 up, 11 in 2023-11-17T00:04:59.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:04:59.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:59 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:04:59.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:59 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:04:59.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:04:59 smithi006 ceph-mon[97840]: osdmap e72: 11 total, 11 up, 11 in 2023-11-17T00:05:00.027 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- 172.21.15.186:0/2363272679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 msgr2=0x7fddfc107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:00.027 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 --2- 172.21.15.186:0/2363272679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc107670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b0a0 tx=0x7fddf400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- 172.21.15.186:0/2363272679 shutdown_connections 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 --2- 172.21.15.186:0/2363272679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc107670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 --2- 172.21.15.186:0/2363272679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc0ff700 0x7fddfc0ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 --2- 172.21.15.186:0/2363272679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc100420 0x7fddfc100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- 172.21.15.186:0/2363272679 >> 172.21.15.186:0/2363272679 conn(0x7fddfc0f9f70 msgr2=0x7fddfc0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- 172.21.15.186:0/2363272679 shutdown_connections 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- 172.21.15.186:0/2363272679 wait complete. 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 Processor -- start 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.022+0000 7fde03c9b700 1 -- start start 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc0ff700 0x7fddfc0fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:00.028 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc100420 0x7fddfc19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc19fe80 con 0x7fddfc0ff700 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc19fff0 con 0x7fddfc100420 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc1a01b0 con 0x7fddfc100dd0 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55950/0 (socket says 172.21.15.186:55950) 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 -- 172.21.15.186:0/3161181466 learned_addr learned my addr 172.21.15.186:0/3161181466 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde01a37700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc0ff700 0x7fddfc0fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde01236700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc100420 0x7fddfc19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 -- 172.21.15.186:0/3161181466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc100420 msgr2=0x7fddfc19d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:00.029 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc100420 0x7fddfc19d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 -- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc0ff700 msgr2=0x7fddfc0fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc0ff700 0x7fddfc0fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 -- 172.21.15.186:0/3161181466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009d00 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde02238700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b780 tx=0x7fddf400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf400e040 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddf4004e60 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc10ab60 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40053b0 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.023+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc10a9d0 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.024+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc066ea0 con 0x7fddfc100dd0 2023-11-17T00:05:00.030 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.025+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fddf4005510 con 0x7fddfc100dd0 2023-11-17T00:05:00.031 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.026+0000 7fddf2ffd700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 0x7fdde80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:00.031 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.026+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fddf40cddd0 con 0x7fddfc100dd0 2023-11-17T00:05:00.031 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.026+0000 7fde01a37700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 0x7fdde80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:00.031 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.027+0000 7fde01a37700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 0x7fdde80a3600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fddfc068120 tx=0x7fddec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:00.031 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.029+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fddf4097be0 con 0x7fddfc100dd0 2023-11-17T00:05:00.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:59 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:05:00.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:59 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:05:00.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:04:59 smithi087 ceph-mon[131182]: osdmap e72: 11 total, 11 up, 11 in 2023-11-17T00:05:00.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:59 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:05:00.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:59 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:05:00.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:04:59 smithi186 ceph-mon[131487]: osdmap e72: 11 total, 11 up, 11 in 2023-11-17T00:05:00.224 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:00.223+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fddfc060f30 con 0x7fdde80a1150 2023-11-17T00:05:00.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:00 smithi006 ceph-mon[97840]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:00.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:05:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:05:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:00 smithi006 ceph-mon[97840]: osdmap e73: 11 total, 11 up, 11 in 2023-11-17T00:05:01.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:00 smithi087 ceph-mon[131182]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:05:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:05:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:00 smithi087 ceph-mon[131182]: osdmap e73: 11 total, 11 up, 11 in 2023-11-17T00:05:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:00 smithi186 ceph-mon[131487]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-17T00:05:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-17T00:05:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:00 smithi186 ceph-mon[131487]: osdmap e73: 11 total, 11 up, 11 in 2023-11-17T00:05:01.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:01 smithi186 ceph-mon[131487]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:01.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:01 smithi006 ceph-mon[97840]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:01 smithi087 ceph-mon[131182]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:02.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:02 smithi006 ceph-mon[97840]: osdmap e74: 11 total, 11 up, 11 in 2023-11-17T00:05:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:02 smithi006 ceph-mon[97840]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:02 smithi087 ceph-mon[131182]: osdmap e74: 11 total, 11 up, 11 in 2023-11-17T00:05:03.122 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:02 smithi087 ceph-mon[131182]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:03.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:02 smithi186 ceph-mon[131487]: osdmap e74: 11 total, 11 up, 11 in 2023-11-17T00:05:03.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:02 smithi186 ceph-mon[131487]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: osdmap e75: 11 total, 11 up, 11 in 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/465866956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]': finished 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: osdmap e76: 12 total, 11 up, 12 in 2023-11-17T00:05:03.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:03.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:03 smithi006 ceph-mon[97840]: from='client.? 172.21.15.186:0/3452761249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:05:04.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: osdmap e75: 11 total, 11 up, 11 in 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/465866956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]': finished 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: osdmap e76: 12 total, 11 up, 12 in 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:03 smithi087 ceph-mon[131182]: from='client.? 172.21.15.186:0/3452761249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:05:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: osdmap e75: 11 total, 11 up, 11 in 2023-11-17T00:05:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/465866956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]: dispatch 2023-11-17T00:05:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f058968e-7426-4a1c-917e-09beb2be51ac"}]': finished 2023-11-17T00:05:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: osdmap e76: 12 total, 11 up, 12 in 2023-11-17T00:05:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:04.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:03 smithi186 ceph-mon[131487]: from='client.? 172.21.15.186:0/3452761249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-17T00:05:04.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:04 smithi006 ceph-mon[97840]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:04 smithi087 ceph-mon[131182]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:04 smithi186 ceph-mon[131487]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:07.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:06 smithi087 ceph-mon[131182]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:05:07.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:06 smithi186 ceph-mon[131487]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:05:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:06 smithi006 ceph-mon[97840]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-11-17T00:05:09.048 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:08 smithi186 ceph-mon[131487]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-17T00:05:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:08 smithi087 ceph-mon[131182]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-17T00:05:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:08 smithi006 ceph-mon[97840]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-17T00:05:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-17T00:05:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:10.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-17T00:05:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:10.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-17T00:05:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:10 smithi087 ceph-mon[131182]: Deploying daemon osd.11 on smithi186 2023-11-17T00:05:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:10 smithi087 ceph-mon[131182]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:05:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:10 smithi186 ceph-mon[131487]: Deploying daemon osd.11 on smithi186 2023-11-17T00:05:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:10 smithi186 ceph-mon[131487]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:05:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:10 smithi006 ceph-mon[97840]: Deploying daemon osd.11 on smithi186 2023-11-17T00:05:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:10 smithi006 ceph-mon[97840]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-17T00:05:12.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:12 smithi186 ceph-mon[131487]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-17T00:05:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:12 smithi087 ceph-mon[131182]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-17T00:05:13.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:12 smithi006 ceph-mon[97840]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-17T00:05:14.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:14 smithi087 ceph-mon[131182]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-17T00:05:14.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:14.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:14.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:14 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:14.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:14 smithi186 ceph-mon[131487]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-17T00:05:14.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:14.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:14.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:14 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:14.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:14 smithi006 ceph-mon[97840]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-17T00:05:14.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:14.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:14.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:14 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:16 smithi087 ceph-mon[131182]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:05:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:16 smithi186 ceph-mon[131487]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:05:17.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:16 smithi006 ceph-mon[97840]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-17T00:05:17.341 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2023-11-17T00:05:17.342 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.340+0000 7fddf2ffd700 1 -- 172.21.15.186:0/3161181466 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fddfc060f30 con 0x7fdde80a1150 2023-11-17T00:05:17.343 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 msgr2=0x7fdde80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:17.343 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 0x7fdde80a3600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fddfc068120 tx=0x7fddec009260 comp rx=0 tx=0).stop 2023-11-17T00:05:17.344 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 msgr2=0x7fddfc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:17.344 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b780 tx=0x7fddf400be10 comp rx=0 tx=0).stop 2023-11-17T00:05:17.410 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 shutdown_connections 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdde80a1150 0x7fdde80a3600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fddfc0ff700 0x7fddfc0fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddfc100dd0 0x7fddfc19f7d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 --2- 172.21.15.186:0/3161181466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddfc100420 0x7fddfc19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 >> 172.21.15.186:0/3161181466 conn(0x7fddfc0f9f70 msgr2=0x7fddfc0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 shutdown_connections 2023-11-17T00:05:17.411 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:17.343+0000 7fde03c9b700 1 -- 172.21.15.186:0/3161181466 wait complete. 2023-11-17T00:05:17.787 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.11.service 2023-11-17T00:05:17.789 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-17T00:05:17.789 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json 2023-11-17T00:05:17.822 INFO:journalctl@ceph.osd.11.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:05:18.092 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:05:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:05:17.997+0000 7f732f93a640 -1 osd.11 0 log_to_monitors true 2023-11-17T00:05:18.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:18.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:18.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:18 smithi006 ceph-mon[97840]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.344 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.345 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:18.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:18 smithi186 ceph-mon[131487]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:18 smithi087 ceph-mon[131182]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-17T00:05:18.662 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:05:19.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:19 smithi006 ceph-mon[97840]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-17T00:05:19.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:19 smithi006 ceph-mon[97840]: osdmap e77: 12 total, 11 up, 12 in 2023-11-17T00:05:19.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:19 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:19.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:19 smithi006 ceph-mon[97840]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.414 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:19 smithi006 ceph-mon[97840]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:19 smithi087 ceph-mon[131182]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-17T00:05:19.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:19 smithi087 ceph-mon[131182]: osdmap e77: 12 total, 11 up, 12 in 2023-11-17T00:05:19.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:19 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:19.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:19 smithi087 ceph-mon[131182]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:19 smithi087 ceph-mon[131182]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-mon[131487]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-17T00:05:19.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-mon[131487]: osdmap e77: 12 total, 11 up, 12 in 2023-11-17T00:05:19.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:19.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-mon[131487]: from='osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-mon[131487]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-11-17T00:05:19.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:05:19.310+0000 7f7324fb1700 -1 osd.11 0 waiting for initial osdmap 2023-11-17T00:05:19.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:05:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:05:19.327+0000 7f731eda2700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-17T00:05:20.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/1177965501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8101e80 msgr2=0x7fcbd8108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:20.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/1177965501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8101e80 0x7fcbd8108740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b0a0 tx=0x7fcbd000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/1177965501 shutdown_connections 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/1177965501 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd8101e80 0x7fcbd8108740 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/1177965501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd81002f0 0x7fcbd8100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/1177965501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 0x7fcbd8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/1177965501 >> 172.21.15.6:0/1177965501 conn(0x7fcbd80fb900 msgr2=0x7fcbd80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/1177965501 shutdown_connections 2023-11-17T00:05:20.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.373+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/1177965501 wait complete. 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.374+0000 7fcbdd9f4700 1 Processor -- start 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.374+0000 7fcbdd9f4700 1 -- start start 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 0x7fcbd81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 0x7fcbd819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:20.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd819dd00 con 0x7fcbd81002f0 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd819de40 con 0x7fcbd81014f0 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbdd9f4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd819dfd0 con 0x7fcbd8101e80 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34766/0 (socket says 172.21.15.6:34766) 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 -- 172.21.15.6:0/2972346734 learned_addr learned my addr 172.21.15.6:0/2972346734 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd77fe700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 0x7fcbd819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd67fc700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 0x7fcbd81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 -- 172.21.15.6:0/2972346734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 msgr2=0x7fcbd819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 0x7fcbd819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 -- 172.21.15.6:0/2972346734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 msgr2=0x7fcbd81966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 0x7fcbd81966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.375+0000 7fcbd6ffd700 1 -- 172.21.15.6:0/2972346734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009d00 con 0x7fcbd81002f0 2023-11-17T00:05:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbd77fe700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 0x7fcbd819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbd6ffd700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e9a0 tx=0x7fcbc800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8009790 con 0x7fcbd81002f0 2023-11-17T00:05:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd810bcb0 con 0x7fcbd81002f0 2023-11-17T00:05:20.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbc800c810 con 0x7fcbd81002f0 2023-11-17T00:05:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8003c70 con 0x7fcbd81002f0 2023-11-17T00:05:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.376+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd810bb20 con 0x7fcbd81002f0 2023-11-17T00:05:20.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.377+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd8107830 con 0x7fcbd81002f0 2023-11-17T00:05:20.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.380+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcbc800cc80 con 0x7fcbd81002f0 2023-11-17T00:05:20.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.382+0000 7fcbdc9f2700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 0x7fcbc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:20.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.382+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcbc8014070 con 0x7fcbd81002f0 2023-11-17T00:05:20.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.382+0000 7fcbd67fc700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 0x7fcbc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:20.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.383+0000 7fcbd67fc700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 0x7fcbc00a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009c60 tx=0x7fcbcc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:20.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.386+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcbc80961c0 con 0x7fcbd81002f0 2023-11-17T00:05:20.476 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:05:20.476 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: osdmap e78: 12 total, 11 up, 12 in 2023-11-17T00:05:20.476 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:20.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.486 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:20.487 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:20 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.594+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcbd802cfe0 con 0x7fcbd81002f0 2023-11-17T00:05:20.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.594+0000 7fcbdc9f2700 1 -- 172.21.15.6:0/2972346734 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fcbc8003dd0 con 0x7fcbd81002f0 2023-11-17T00:05:20.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 msgr2=0x7fcbc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 0x7fcbc00a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009c60 tx=0x7fcbcc009260 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 msgr2=0x7fcbd8196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e9a0 tx=0x7fcbc800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 shutdown_connections 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcbc00a1150 0x7fcbc00a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd81002f0 0x7fcbd8196170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbd8101e80 0x7fcbd819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 --2- 172.21.15.6:0/2972346734 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd81014f0 0x7fcbd81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 >> 172.21.15.6:0/2972346734 conn(0x7fcbd80fb900 msgr2=0x7fcbd8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 shutdown_connections 2023-11-17T00:05:20.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:20.597+0000 7fcbdd9f4700 1 -- 172.21.15.6:0/2972346734 wait complete. 2023-11-17T00:05:20.600 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:05:20.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: osdmap e78: 12 total, 11 up, 12 in 2023-11-17T00:05:20.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:20.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.609 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.609 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.609 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:20.610 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:20 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: osdmap e78: 12 total, 11 up, 12 in 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-17T00:05:20.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:20.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:20 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:21.051 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1700179520,"num_in_osds":12,"osd_in_since":1700179502,"num_remapped_pgs":0} 2023-11-17T00:05:21.053 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-17T00:05:21.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: purged_snaps scrub starts 2023-11-17T00:05:21.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: purged_snaps scrub ok 2023-11-17T00:05:21.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: Detected new or changed devices on smithi186 2023-11-17T00:05:21.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 4292M 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] boot 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: osdmap e79: 12 total, 12 up, 12 in 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:21 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2972346734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: purged_snaps scrub starts 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: purged_snaps scrub ok 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: Detected new or changed devices on smithi186 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 4292M 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:05:21.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] boot 2023-11-17T00:05:21.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: osdmap e79: 12 total, 12 up, 12 in 2023-11-17T00:05:21.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:21 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2972346734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-17T00:05:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: purged_snaps scrub starts 2023-11-17T00:05:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: purged_snaps scrub ok 2023-11-17T00:05:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: Detected new or changed devices on smithi186 2023-11-17T00:05:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 4292M 2023-11-17T00:05:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: osd.11 [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] boot 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: osdmap e79: 12 total, 12 up, 12 in 2023-11-17T00:05:21.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:05:21.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:21 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2972346734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-17T00:05:21.922 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:05:22.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:22 smithi006 ceph-mon[97840]: osdmap e80: 12 total, 12 up, 12 in 2023-11-17T00:05:22.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:22 smithi006 ceph-mon[97840]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:22.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:22 smithi087 ceph-mon[131182]: osdmap e80: 12 total, 12 up, 12 in 2023-11-17T00:05:22.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:22 smithi087 ceph-mon[131182]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:22.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:22 smithi186 ceph-mon[131487]: osdmap e80: 12 total, 12 up, 12 in 2023-11-17T00:05:22.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:22 smithi186 ceph-mon[131487]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 -- 172.21.15.6:0/1603655547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401055e0 msgr2=0x7f934010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 --2- 172.21.15.6:0/1603655547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401055e0 0x7f934010d160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f933800b0d0 tx=0x7f933800b3e0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 -- 172.21.15.6:0/1603655547 shutdown_connections 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 --2- 172.21.15.6:0/1603655547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93401055e0 0x7f934010d160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 --2- 172.21.15.6:0/1603655547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9340100540 0x7f9340100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 --2- 172.21.15.6:0/1603655547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 0x7f9340104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 -- 172.21.15.6:0/1603655547 >> 172.21.15.6:0/1603655547 conn(0x7f93400fbdc0 msgr2=0x7f93400fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:23.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 -- 172.21.15.6:0/1603655547 shutdown_connections 2023-11-17T00:05:23.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.644+0000 7f93484e6700 1 -- 172.21.15.6:0/1603655547 wait complete. 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.645+0000 7f93484e6700 1 Processor -- start 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.645+0000 7f93484e6700 1 -- start start 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.645+0000 7f93484e6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f93484e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 0x7f9340198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f93484e6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93401055e0 0x7f934019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f93484e6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019c5c0 con 0x7f9340100540 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f93484e6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019c730 con 0x7f93401055e0 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f93484e6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019c8f0 con 0x7f9340100ee0 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34774/0 (socket says 172.21.15.6:34774) 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 -- 172.21.15.6:0/2033038701 learned_addr learned my addr 172.21.15.6:0/2033038701 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9345a81700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 0x7f9340198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346a83700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93401055e0 0x7f934019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 -- 172.21.15.6:0/2033038701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 msgr2=0x7f9340198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 0x7f9340198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 -- 172.21.15.6:0/2033038701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93401055e0 msgr2=0x7f934019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93401055e0 0x7f934019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.646+0000 7f9346282700 1 -- 172.21.15.6:0/2033038701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009d00 con 0x7f9340100540 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f9346282700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f933c00d950 tx=0x7f933c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00f890 con 0x7f9340100540 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933c010460 con 0x7f9340100540 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9340110370 con 0x7f9340100540 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00e610 con 0x7f9340100540 2023-11-17T00:05:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.647+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401101e0 con 0x7f9340100540 2023-11-17T00:05:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.648+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9340110500 con 0x7f9340100540 2023-11-17T00:05:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.650+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f933c0108d0 con 0x7f9340100540 2023-11-17T00:05:23.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.651+0000 7f93377fe700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 0x7f932c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.652+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f933c0cbc80 con 0x7f9340100540 2023-11-17T00:05:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.652+0000 7f9345a81700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 0x7f932c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.652+0000 7f9345a81700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 0x7f932c0a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9340199ce0 tx=0x7f933000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:23.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.656+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f933c0957d0 con 0x7f9340100540 2023-11-17T00:05:23.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.864+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93401a57d0 con 0x7f9340100540 2023-11-17T00:05:23.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.865+0000 7f93377fe700 1 -- 172.21.15.6:0/2033038701 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7f933c098df0 con 0x7f9340100540 2023-11-17T00:05:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":80,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:05:21.318020+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:05:23.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 msgr2=0x7f932c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:23.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 0x7f932c0a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9340199ce0 tx=0x7f933000b410 comp rx=0 tx=0).stop 2023-11-17T00:05:23.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 msgr2=0x7f9340198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:23.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f933c00d950 tx=0x7f933c00dc60 comp rx=0 tx=0).stop 2023-11-17T00:05:23.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 shutdown_connections 2023-11-17T00:05:23.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f932c0a11d0 0x7f932c0a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9340100540 0x7f9340198710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93401055e0 0x7f934019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 --2- 172.21.15.6:0/2033038701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9340100ee0 0x7f9340198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 >> 172.21.15.6:0/2033038701 conn(0x7f93400fbdc0 msgr2=0x7f934010b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 shutdown_connections 2023-11-17T00:05:23.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:23.867+0000 7f93484e6700 1 -- 172.21.15.6:0/2033038701 wait complete. 2023-11-17T00:05:24.312 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-17T00:05:24.312 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-11-17T00:05:24.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:24 smithi087 ceph-mon[131182]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:24.992 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:24 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2033038701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:24.993 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:24 smithi006 ceph-mon[97840]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:24.993 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:24 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2033038701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:24.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:24 smithi186 ceph-mon[131487]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:24.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:24 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2033038701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:25.174 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2034944942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 msgr2=0x7f62bc10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2034944942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc10d160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b0d0 tx=0x7f62b400b3e0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2034944942 shutdown_connections 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2034944942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc10d160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2034944942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100540 0x7f62bc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2034944942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100ee0 0x7f62bc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2034944942 >> 172.21.15.6:0/2034944942 conn(0x7f62bc0fbdc0 msgr2=0x7f62bc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:26.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2034944942 shutdown_connections 2023-11-17T00:05:26.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.679+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2034944942 wait complete. 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.680+0000 7f62c3bdf700 1 Processor -- start 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.680+0000 7f62c3bdf700 1 -- start start 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.680+0000 7f62c3bdf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c3bdf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100ee0 0x7f62bc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c3bdf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c3bdf700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc073c20 con 0x7f62bc1055e0 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c3bdf700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1a4c60 con 0x7f62bc100ee0 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c3bdf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1a4dd0 con 0x7f62bc100540 2023-11-17T00:05:26.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c217c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45044/0 (socket says 172.21.15.6:45044) 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 -- 172.21.15.6:0/2481149144 learned_addr learned my addr 172.21.15.6:0/2481149144 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c117a700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100ee0 0x7f62bc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 -- 172.21.15.6:0/2481149144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100ee0 msgr2=0x7f62bc073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100ee0 0x7f62bc073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 -- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 msgr2=0x7f62bc1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c197b700 1 -- 172.21.15.6:0/2481149144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4009d00 con 0x7f62bc100540 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.681+0000 7f62c217c700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:26.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62c197b700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62b800e9a0 tx=0x7f62b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8009790 con 0x7f62bc100540 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b8010430 con 0x7f62bc100540 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8018680 con 0x7f62bc100540 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc110370 con 0x7f62bc100540 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.682+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc1101e0 con 0x7f62bc100540 2023-11-17T00:05:26.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.683+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc110500 con 0x7f62bc100540 2023-11-17T00:05:26.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.685+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f62b80105a0 con 0x7f62bc100540 2023-11-17T00:05:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.686+0000 7f62b2ffd700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 0x7f62a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.687+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62b8014070 con 0x7f62bc100540 2023-11-17T00:05:26.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.687+0000 7f62c117a700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 0x7f62a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:26.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.687+0000 7f62c117a700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 0x7f62a80a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0745e0 tx=0x7f62ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:26.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.689+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f62b8096350 con 0x7f62bc100540 2023-11-17T00:05:26.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.897+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62bc1a57b0 con 0x7f62bc100540 2023-11-17T00:05:26.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.898+0000 7f62b2ffd700 1 -- 172.21.15.6:0/2481149144 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62b8099970 con 0x7f62bc100540 2023-11-17T00:05:26.899 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 msgr2=0x7f62a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 0x7f62a80a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0745e0 tx=0x7f62ac009260 comp rx=0 tx=0).stop 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 msgr2=0x7f62bc072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62b800e9a0 tx=0x7f62b800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 shutdown_connections 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f62a80a1150 0x7f62a80a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f62bc1055e0 0x7f62bc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62bc100540 0x7f62bc072df0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 --2- 172.21.15.6:0/2481149144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62bc100ee0 0x7f62bc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 >> 172.21.15.6:0/2481149144 conn(0x7f62bc0fbdc0 msgr2=0x7f62bc0fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 shutdown_connections 2023-11-17T00:05:26.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:26.901+0000 7f62c3bdf700 1 -- 172.21.15.6:0/2481149144 wait complete. 2023-11-17T00:05:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:26 smithi087 ceph-mon[131182]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:27.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:26 smithi006 ceph-mon[97840]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:26 smithi186 ceph-mon[131487]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:27.461 INFO:tasks.cephadm:Adding mds.a on smithi006 2023-11-17T00:05:27.461 INFO:tasks.cephadm:Adding mds.d on smithi006 2023-11-17T00:05:27.461 INFO:tasks.cephadm:Adding mds.g on smithi006 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.j on smithi006 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.b on smithi087 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.e on smithi087 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.h on smithi087 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.k on smithi087 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.c on smithi186 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.f on smithi186 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.i on smithi186 2023-11-17T00:05:27.462 INFO:tasks.cephadm:Adding mds.l on smithi186 2023-11-17T00:05:27.462 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph orch apply mds all '12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2023-11-17T00:05:28.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:27 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2481149144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:28.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:27 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2481149144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:28.118 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:05:28.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:27 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2481149144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:28.933 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:28 smithi186 ceph-mon[131487]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:29.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:28 smithi087 ceph-mon[131182]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:29.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:28 smithi006 ceph-mon[97840]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 -- 172.21.15.186:0/4039150751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 msgr2=0x7f3c08108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/4039150751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c08108410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000b0a0 tx=0x7f3c0000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 -- 172.21.15.186:0/4039150751 shutdown_connections 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/4039150751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c08108410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/4039150751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c08100dd0 0x7f3c081031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/4039150751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c081036f0 0x7f3c08105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 -- 172.21.15.186:0/4039150751 >> 172.21.15.186:0/4039150751 conn(0x7f3c080fa7f0 msgr2=0x7f3c080fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 -- 172.21.15.186:0/4039150751 shutdown_connections 2023-11-17T00:05:29.461 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.459+0000 7f3c0f70f700 1 -- 172.21.15.186:0/4039150751 wait complete. 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 Processor -- start 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 -- start start 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 0x7f3c081961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 0x7f3c08196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0819dd70 con 0x7f3c08100dd0 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0819deb0 con 0x7f3c081036f0 2023-11-17T00:05:29.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0f70f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0819e040 con 0x7f3c08106010 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:56076/0 (socket says 172.21.15.186:56076) 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 -- 172.21.15.186:0/675229364 learned_addr learned my addr 172.21.15.186:0/675229364 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0d4ab700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 0x7f3c081961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0ccaa700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 0x7f3c08196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 -- 172.21.15.186:0/675229364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 msgr2=0x7f3c08196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 0x7f3c08196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 -- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 msgr2=0x7f3c081961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 0x7f3c081961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.469 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 -- 172.21.15.186:0/675229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c00009d00 con 0x7f3c08106010 2023-11-17T00:05:29.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0ccaa700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 0x7f3c08196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:05:29.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0d4ab700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 0x7f3c081961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:05:29.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.460+0000 7f3c0dcac700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006070 tx=0x7f3c00009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:29.470 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.461+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000e040 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.461+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c00004e90 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.461+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001d6e0 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.461+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0810bcb0 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.461+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0810bb20 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.462+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c080fc360 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.463+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3c00003a00 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.463+0000 7f3bfe7fc700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 0x7f3bf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.463+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3c000ce2c0 con 0x7f3c08106010 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.463+0000 7f3c0d4ab700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 0x7f3bf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.464+0000 7f3c0d4ab700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 0x7f3bf40a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005950 tx=0x7f3bf80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:29.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.466+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3c00097f40 con 0x7f3c08106010 2023-11-17T00:05:29.663 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.662+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c08061be0 con 0x7f3bf40a1150 2023-11-17T00:05:29.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.671+0000 7f3bfe7fc700 1 -- 172.21.15.186:0/675229364 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3c08061be0 con 0x7f3bf40a1150 2023-11-17T00:05:29.672 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2023-11-17T00:05:29.674 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 msgr2=0x7f3bf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:29.674 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 0x7f3bf40a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005950 tx=0x7f3bf80058e0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.674 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 msgr2=0x7f3c0819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006070 tx=0x7f3c00009590 comp rx=0 tx=0).stop 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 shutdown_connections 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3bf40a1150 0x7f3bf40a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c08100dd0 0x7f3c081961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c08106010 0x7f3c0819d7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 --2- 172.21.15.186:0/675229364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c081036f0 0x7f3c08196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 >> 172.21.15.186:0/675229364 conn(0x7f3c080fa7f0 msgr2=0x7f3c08101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 shutdown_connections 2023-11-17T00:05:29.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-11-17T00:05:29.673+0000 7f3c0f70f700 1 -- 172.21.15.186:0/675229364 wait complete. 2023-11-17T00:05:30.168 DEBUG:teuthology.orchestra.run.smithi006:mds.a> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.a.service 2023-11-17T00:05:30.175 DEBUG:teuthology.orchestra.run.smithi006:mds.d> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.d.service 2023-11-17T00:05:30.220 DEBUG:teuthology.orchestra.run.smithi006:mds.g> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.g.service 2023-11-17T00:05:30.257 INFO:journalctl@ceph.mds.a.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:05:30.261 DEBUG:teuthology.orchestra.run.smithi006:mds.j> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.j.service 2023-11-17T00:05:30.298 INFO:journalctl@ceph.mds.d.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:05:30.300 DEBUG:teuthology.orchestra.run.smithi087:mds.b> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.b.service 2023-11-17T00:05:30.303 DEBUG:teuthology.orchestra.run.smithi087:mds.e> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.e.service 2023-11-17T00:05:30.334 INFO:journalctl@ceph.mds.g.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:05:30.342 INFO:journalctl@ceph.mds.b.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:05:30.343 DEBUG:teuthology.orchestra.run.smithi087:mds.h> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service 2023-11-17T00:05:30.370 INFO:journalctl@ceph.mds.j.smithi006.stdout:-- Logs begin at Thu 2023-11-16 23:40:09 UTC. -- 2023-11-17T00:05:30.371 INFO:journalctl@ceph.mds.e.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:05:30.373 DEBUG:teuthology.orchestra.run.smithi087:mds.k> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.k.service 2023-11-17T00:05:30.399 INFO:journalctl@ceph.mds.h.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:05:30.401 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.c.service 2023-11-17T00:05:30.403 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.f.service 2023-11-17T00:05:30.427 INFO:journalctl@ceph.mds.k.smithi087.stdout:-- Logs begin at Thu 2023-11-16 23:36:21 UTC. -- 2023-11-17T00:05:30.444 INFO:journalctl@ceph.mds.c.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:05:30.445 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.i.service 2023-11-17T00:05:30.471 INFO:journalctl@ceph.mds.f.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:05:30.472 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.l.service 2023-11-17T00:05:30.497 INFO:journalctl@ceph.mds.i.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:05:30.498 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-17T00:05:30.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:05:30.523 INFO:journalctl@ceph.mds.l.smithi186.stdout:-- Logs begin at Thu 2023-11-16 23:36:41 UTC. -- 2023-11-17T00:05:30.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:30.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-11-17T00:05:30.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:30.672 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 -- 172.21.15.6:0/553697369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f649410c8a0 msgr2=0x7f649410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 --2- 172.21.15.6:0/553697369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f649410c8a0 0x7f649410ec80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b0a0 tx=0x7f648c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 -- 172.21.15.6:0/553697369 shutdown_connections 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 --2- 172.21.15.6:0/553697369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f649410c8a0 0x7f649410ec80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 --2- 172.21.15.6:0/553697369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6494074b30 0x7f6494072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 --2- 172.21.15.6:0/553697369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494073550 0x7f64940739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 -- 172.21.15.6:0/553697369 >> 172.21.15.6:0/553697369 conn(0x7f64940fb8b0 msgr2=0x7f64940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 -- 172.21.15.6:0/553697369 shutdown_connections 2023-11-17T00:05:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.717+0000 7f6498dff700 1 -- 172.21.15.6:0/553697369 wait complete. 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 Processor -- start 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 -- start start 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 0x7f649419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f649410c8a0 0x7f64941a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a42f0 con 0x7f6494073550 2023-11-17T00:05:30.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a4430 con 0x7f6494074b30 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.718+0000 7f6498dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a45c0 con 0x7f649410c8a0 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55280/0 (socket says 172.21.15.6:55280) 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 -- 172.21.15.6:0/2365108158 learned_addr learned my addr 172.21.15.6:0/2365108158 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f6492d9d700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f649410c8a0 0x7f64941a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f6491d9b700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 0x7f649419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 -- 172.21.15.6:0/2365108158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f649410c8a0 msgr2=0x7f64941a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f649410c8a0 0x7f64941a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 -- 172.21.15.6:0/2365108158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 msgr2=0x7f649419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 0x7f649419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 -- 172.21.15.6:0/2365108158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c009d00 con 0x7f6494073550 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f6491d9b700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 0x7f649419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:05:30.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f649259c700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f648400d980 tx=0x7f648400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484009930 con 0x7f6494073550 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6484010460 con 0x7f6494073550 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648400f620 con 0x7f6494073550 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.719+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64941122b0 con 0x7f6494073550 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.720+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6494112120 con 0x7f6494073550 2023-11-17T00:05:30.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.721+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64941023f0 con 0x7f6494073550 2023-11-17T00:05:30.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.722+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f64840108d0 con 0x7f6494073550 2023-11-17T00:05:30.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.724+0000 7f64837fe700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 0x7f647c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:30.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.724+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64840cbf50 con 0x7f6494073550 2023-11-17T00:05:30.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.724+0000 7f6491d9b700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 0x7f647c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:30.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.725+0000 7f6491d9b700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 0x7f647c0a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6494074850 tx=0x7f648800b670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:30.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.728+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6484095aa0 con 0x7f6494073550 2023-11-17T00:05:30.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.936+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f649402cfe0 con 0x7f6494073550 2023-11-17T00:05:30.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.937+0000 7f64837fe700 1 -- 172.21.15.6:0/2365108158 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7f64840990c0 con 0x7f6494073550 2023-11-17T00:05:30.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:30.938 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":80,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:05:21.318020+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:05:30.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 msgr2=0x7f647c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 0x7f647c0a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6494074850 tx=0x7f648800b670 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 msgr2=0x7f649419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f648400d980 tx=0x7f648400dc90 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 shutdown_connections 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f647c0a1150 0x7f647c0a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6494073550 0x7f649419c770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f649410c8a0 0x7f64941a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 --2- 172.21.15.6:0/2365108158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6494074b30 0x7f649419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 >> 172.21.15.6:0/2365108158 conn(0x7f64940fb8b0 msgr2=0x7f64940fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 shutdown_connections 2023-11-17T00:05:30.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:30.940+0000 7f6498dff700 1 -- 172.21.15.6:0/2365108158 wait complete. 2023-11-17T00:05:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-11-17T00:05:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: Deploying daemon mds.l on smithi186 2023-11-17T00:05:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:30 smithi006 ceph-mon[97840]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:30.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-17T00:05:30.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:05:30.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:30.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:30.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:30.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:30.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:30.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: Deploying daemon mds.l on smithi186 2023-11-17T00:05:30.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:30 smithi186 ceph-mon[131487]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Saving service mds.all spec with placement smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:31.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: Deploying daemon mds.l on smithi186 2023-11-17T00:05:31.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:30 smithi087 ceph-mon[131182]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:31.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 -- 172.21.15.6:0/588757409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c4101490 msgr2=0x7f28c41018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 --2- 172.21.15.6:0/588757409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c4101490 0x7f28c41018e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009990 tx=0x7f28b4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 -- 172.21.15.6:0/588757409 shutdown_connections 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 --2- 172.21.15.6:0/588757409 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c41002a0 0x7f28c41006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 --2- 172.21.15.6:0/588757409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c41086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 --2- 172.21.15.6:0/588757409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c4101490 0x7f28c41018e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 -- 172.21.15.6:0/588757409 >> 172.21.15.6:0/588757409 conn(0x7f28c40fb880 msgr2=0x7f28c40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:31.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 -- 172.21.15.6:0/588757409 shutdown_connections 2023-11-17T00:05:31.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.167+0000 7f28cbbff700 1 -- 172.21.15.6:0/588757409 wait complete. 2023-11-17T00:05:31.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.168+0000 7f28cbbff700 1 Processor -- start 2023-11-17T00:05:31.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.168+0000 7f28cbbff700 1 -- start start 2023-11-17T00:05:31.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.168+0000 7f28cbbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.168+0000 7f28cbbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4101490 0x7f28c4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28cbbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28cbbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4073bc0 con 0x7f28c4101490 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28cbbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4073d30 con 0x7f28c4101e20 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28cbbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c419ff60 con 0x7f28c41002a0 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c919a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4101490 0x7f28c4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55788/0 (socket says 172.21.15.6:55788) 2023-11-17T00:05:31.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 -- 172.21.15.6:0/1804176814 learned_addr learned my addr 172.21.15.6:0/1804176814 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:31.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28ca19c700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 -- 172.21.15.6:0/1804176814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 msgr2=0x7f28c4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 -- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4101490 msgr2=0x7f28c4072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4101490 0x7f28c4072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 -- 172.21.15.6:0/1804176814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b40095f0 con 0x7f28c41002a0 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28ca19c700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.169+0000 7f28c999b700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28c000e970 tx=0x7f28c000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:31.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.170+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0009790 con 0x7f28c41002a0 2023-11-17T00:05:31.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.170+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28c000eec0 con 0x7f28c41002a0 2023-11-17T00:05:31.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.170+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c410bba0 con 0x7f28c41002a0 2023-11-17T00:05:31.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.170+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003cf0 con 0x7f28c41002a0 2023-11-17T00:05:31.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.170+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c410ba10 con 0x7f28c41002a0 2023-11-17T00:05:31.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.171+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4107220 con 0x7f28c41002a0 2023-11-17T00:05:31.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.173+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f28c000ccd0 con 0x7f28c41002a0 2023-11-17T00:05:31.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.174+0000 7f28baffd700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 0x7f28b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.174+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28c0014070 con 0x7f28c41002a0 2023-11-17T00:05:31.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.174+0000 7f28c919a700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 0x7f28b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.175+0000 7f28c919a700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 0x7f28b00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28b400b680 tx=0x7f28b4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:31.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.176+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28c0096880 con 0x7f28c41002a0 2023-11-17T00:05:31.366 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 systemd[1]: Starting Ceph mds.l for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:31.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.388+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28c4061700 con 0x7f28c41002a0 2023-11-17T00:05:31.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.389+0000 7f28baffd700 1 -- 172.21.15.6:0/1804176814 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28c0099ea0 con 0x7f28c41002a0 2023-11-17T00:05:31.389 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:05:31.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.391+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 msgr2=0x7f28b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.391+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 0x7f28b00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28b400b680 tx=0x7f28b4005960 comp rx=0 tx=0).stop 2023-11-17T00:05:31.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.391+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 msgr2=0x7f28c40748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.391+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28c000e970 tx=0x7f28c000ec80 comp rx=0 tx=0).stop 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 shutdown_connections 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f28b00a1150 0x7f28b00a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28c4101e20 0x7f28c4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28c41002a0 0x7f28c40748a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 --2- 172.21.15.6:0/1804176814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4101490 0x7f28c4072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 >> 172.21.15.6:0/1804176814 conn(0x7f28c40fb880 msgr2=0x7f28c40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 shutdown_connections 2023-11-17T00:05:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.392+0000 7f28cbbff700 1 -- 172.21.15.6:0/1804176814 wait complete. 2023-11-17T00:05:31.413 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 -- 172.21.15.6:0/3005359676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 msgr2=0x7f979c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 --2- 172.21.15.6:0/3005359676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c0739c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9784009990 tx=0x7f9784009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 -- 172.21.15.6:0/3005359676 shutdown_connections 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 --2- 172.21.15.6:0/3005359676 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c074b30 0x7f979c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 --2- 172.21.15.6:0/3005359676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c108190 0x7f979c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 --2- 172.21.15.6:0/3005359676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c0739c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 -- 172.21.15.6:0/3005359676 >> 172.21.15.6:0/3005359676 conn(0x7f979c0fb8b0 msgr2=0x7f979c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 -- 172.21.15.6:0/3005359676 shutdown_connections 2023-11-17T00:05:31.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.619+0000 7f97a09ff700 1 -- 172.21.15.6:0/3005359676 wait complete. 2023-11-17T00:05:31.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 Processor -- start 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 -- start start 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c074b30 0x7f979c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 0x7f979c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.620+0000 7f97a09ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19fda0 con 0x7f979c108190 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f97a09ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19fee0 con 0x7f979c074b30 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f97a09ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c1a0070 con 0x7f979c073550 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:55798/0 (socket says 172.21.15.6:55798) 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 -- 172.21.15.6:0/2638563354 learned_addr learned my addr 172.21.15.6:0/2638563354 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979ad9d700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 0x7f979c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f9793fff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c074b30 0x7f979c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 -- 172.21.15.6:0/2638563354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c074b30 msgr2=0x7f979c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c074b30 0x7f979c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 -- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 msgr2=0x7f979c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 0x7f979c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 -- 172.21.15.6:0/2638563354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c009520 con 0x7f979c073550 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979ad9d700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 0x7f979c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f979a59c700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f978c00ea30 tx=0x7f978c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.621+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c009820 con 0x7f979c073550 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.622+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978c0044e0 con 0x7f979c073550 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.622+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c003e80 con 0x7f979c073550 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.622+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97840095f0 con 0x7f979c073550 2023-11-17T00:05:31.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.622+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c10de60 con 0x7f979c073550 2023-11-17T00:05:31.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.623+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c106840 con 0x7f979c073550 2023-11-17T00:05:31.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.625+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f978c004000 con 0x7f979c073550 2023-11-17T00:05:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.626+0000 7f97937fe700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 0x7f97880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:31.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.626+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f978c014070 con 0x7f979c073550 2023-11-17T00:05:31.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.626+0000 7f9793fff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 0x7f97880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:31.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.627+0000 7f9793fff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 0x7f97880a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9784006010 tx=0x7f978400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:31.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.630+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f978c096570 con 0x7f979c073550 2023-11-17T00:05:31.793 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:31 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2365108158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:31.793 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:31 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1804176814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:31.793 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 podman[156451]: 2023-11-17T00:05:31.793 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 bash[156451]: 9351d0e622df9da5ad594b21b1e12c2e334a7ef2ef3416635963ac3ee200f23c 2023-11-17T00:05:31.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.884+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979c061810 con 0x7f979c073550 2023-11-17T00:05:31.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.884+0000 7f97937fe700 1 -- 172.21.15.6:0/2638563354 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f978c099b90 con 0x7f979c073550 2023-11-17T00:05:31.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:31.886 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-17T00:05:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 msgr2=0x7f97880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 0x7f97880a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9784006010 tx=0x7f978400b260 comp rx=0 tx=0).stop 2023-11-17T00:05:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 msgr2=0x7f979c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:31.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f978c00ea30 tx=0x7f978c00edf0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 shutdown_connections 2023-11-17T00:05:31.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f97880a11d0 0x7f97880a3680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f979c108190 0x7f979c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f979c073550 0x7f979c198220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 --2- 172.21.15.6:0/2638563354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f979c074b30 0x7f979c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 >> 172.21.15.6:0/2638563354 conn(0x7f979c0fb8b0 msgr2=0x7f979c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 shutdown_connections 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:31.887+0000 7f97a09ff700 1 -- 172.21.15.6:0/2638563354 wait complete. 2023-11-17T00:05:31.898 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 2 2023-11-17T00:05:31.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:31 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2365108158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:31.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:31 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1804176814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:32.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:31 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2365108158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:05:32.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:31 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1804176814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:05:32.115 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 ceph-mds[156491]: -- 172.21.15.186:0/3506600412 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a7ffa3520 con 0x559a7ff79000 2023-11-17T00:05:32.116 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-l[156467]: starting mds.l at 2023-11-17T00:05:32.116 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:05:31 smithi186 systemd[1]: Started Ceph mds.l for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:32.869 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.186:6832/3460267668,v1:172.21.15.186:6833/3460267668] up:boot 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: fsmap 1 up:standby 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2638563354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:32.870 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:32 smithi087 ceph-mon[131182]: Deploying daemon mds.k on smithi087 2023-11-17T00:05:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.186:6832/3460267668,v1:172.21.15.186:6833/3460267668] up:boot 2023-11-17T00:05:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: fsmap 1 up:standby 2023-11-17T00:05:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2638563354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:33.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:32 smithi186 ceph-mon[131487]: Deploying daemon mds.k on smithi087 2023-11-17T00:05:33.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.186:6832/3460267668,v1:172.21.15.186:6833/3460267668] up:boot 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: fsmap 1 up:standby 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2638563354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:32 smithi006 ceph-mon[97840]: Deploying daemon mds.k on smithi087 2023-11-17T00:05:33.568 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 systemd[1]: Starting Ceph mds.k for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:33.851 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 podman[155428]: 2023-11-17T00:05:33.911 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 -- 172.21.15.6:0/1694403494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 msgr2=0x7f06ec1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1694403494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec1083e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009f90 tx=0x7f06e400b330 comp rx=0 tx=0).stop 2023-11-17T00:05:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 -- 172.21.15.6:0/1694403494 shutdown_connections 2023-11-17T00:05:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1694403494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec100d40 0x7f06ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1694403494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec1083e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1694403494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec103660 0x7f06ec105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.128+0000 7f06f3fff700 1 -- 172.21.15.6:0/1694403494 >> 172.21.15.6:0/1694403494 conn(0x7f06ec0fa7a0 msgr2=0x7f06ec0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.129+0000 7f06f3fff700 1 -- 172.21.15.6:0/1694403494 shutdown_connections 2023-11-17T00:05:34.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.129+0000 7f06f3fff700 1 -- 172.21.15.6:0/1694403494 wait complete. 2023-11-17T00:05:34.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 Processor -- start 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 -- start start 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec100d40 0x7f06ec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 0x7f06ec198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec19fec0 con 0x7f06ec103660 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec1a0000 con 0x7f06ec105f80 2023-11-17T00:05:34.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec1a0190 con 0x7f06ec100d40 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.130+0000 7f06f159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 0x7f06ec198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 0x7f06ec198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55338/0 (socket says 172.21.15.6:55338) 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f159a700 1 -- 172.21.15.6:0/1075995414 learned_addr learned my addr 172.21.15.6:0/1075995414 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f1d9b700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec100d40 0x7f06ec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 -- 172.21.15.6:0/1075995414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec100d40 msgr2=0x7f06ec198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec100d40 0x7f06ec198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 -- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 msgr2=0x7f06ec198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 0x7f06ec198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 -- 172.21.15.6:0/1075995414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80095f0 con 0x7f06ec105f80 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f259c700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec19f8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06e400b6c0 tx=0x7f06e4004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e400e040 con 0x7f06ec105f80 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e4004e50 con 0x7f06ec105f80 2023-11-17T00:05:34.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40052a0 con 0x7f06ec105f80 2023-11-17T00:05:34.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.131+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e4009c40 con 0x7f06ec105f80 2023-11-17T00:05:34.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.132+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec10bc70 con 0x7f06ec105f80 2023-11-17T00:05:34.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.133+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ec10be00 con 0x7f06ec105f80 2023-11-17T00:05:34.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.135+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f06e401d470 con 0x7f06ec105f80 2023-11-17T00:05:34.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.137+0000 7f06e2ffd700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 0x7f06d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:34.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.137+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06e40cda50 con 0x7f06ec105f80 2023-11-17T00:05:34.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.137+0000 7f06f1d9b700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 0x7f06d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:34.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.137+0000 7f06f1d9b700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 0x7f06d80a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005fd0 tx=0x7f06e8009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:34.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.140+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f06e4097630 con 0x7f06ec105f80 2023-11-17T00:05:34.350 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 bash[155428]: 5ef2c27fd963032980812c63604998672bddf595fcab3292c377b8a6d44bc1af 2023-11-17T00:05:34.351 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 ceph-mds[155468]: -- 172.21.15.87:0/3573641624 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bca1569520 con 0x55bca153ec00 2023-11-17T00:05:34.351 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-k[155444]: starting mds.k at 2023-11-17T00:05:34.351 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:05:33 smithi087 systemd[1]: Started Ceph mds.k for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:34.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.399+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06ec02cfe0 con 0x7f06ec105f80 2023-11-17T00:05:34.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.400+0000 7f06e2ffd700 1 -- 172.21.15.6:0/1075995414 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f06e409ac50 con 0x7f06ec105f80 2023-11-17T00:05:34.400 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:34.400 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-17T00:05:34.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 msgr2=0x7f06d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:34.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 0x7f06d80a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005fd0 tx=0x7f06e8009310 comp rx=0 tx=0).stop 2023-11-17T00:05:34.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 msgr2=0x7f06ec19f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:34.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec19f8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06e400b6c0 tx=0x7f06e4004c60 comp rx=0 tx=0).stop 2023-11-17T00:05:34.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 shutdown_connections 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06d80a11d0 0x7f06d80a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06ec105f80 0x7f06ec19f8f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06ec100d40 0x7f06ec198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 --2- 172.21.15.6:0/1075995414 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06ec103660 0x7f06ec198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 >> 172.21.15.6:0/1075995414 conn(0x7f06ec0fa7a0 msgr2=0x7f06ec0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.402+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 shutdown_connections 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:34.403+0000 7f06f3fff700 1 -- 172.21.15.6:0/1075995414 wait complete. 2023-11-17T00:05:34.405 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2023-11-17T00:05:35.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:35.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.178 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6832/2959279613,v1:172.21.15.87:6833/2959279613] up:boot 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: fsmap 2 up:standby 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:05:35.179 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:34 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1075995414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6832/2959279613,v1:172.21.15.87:6833/2959279613] up:boot 2023-11-17T00:05:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: fsmap 2 up:standby 2023-11-17T00:05:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:05:35.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:34 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1075995414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:35.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6832/2959279613,v1:172.21.15.87:6833/2959279613] up:boot 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: fsmap 2 up:standby 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:05:35.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:34 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1075995414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:35.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:35 smithi006 ceph-mon[97840]: Deploying daemon mds.j on smithi006 2023-11-17T00:05:36.346 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:36 smithi006 systemd[1]: Starting Ceph mds.j for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:36.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:35 smithi087 ceph-mon[131182]: Deploying daemon mds.j on smithi006 2023-11-17T00:05:36.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:35 smithi186 ceph-mon[131487]: Deploying daemon mds.j on smithi006 2023-11-17T00:05:36.423 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:36.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.647+0000 7ff21ebff700 1 -- 172.21.15.6:0/3145561329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 msgr2=0x7ff2181031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.647+0000 7ff21ebff700 1 --2- 172.21.15.6:0/3145561329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff2181031e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009990 tx=0x7ff20c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 -- 172.21.15.6:0/3145561329 shutdown_connections 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 --2- 172.21.15.6:0/3145561329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218106040 0x7ff218108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 --2- 172.21.15.6:0/3145561329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff2181031e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 --2- 172.21.15.6:0/3145561329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218103720 0x7ff218105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 -- 172.21.15.6:0/3145561329 >> 172.21.15.6:0/3145561329 conn(0x7ff2180fa7c0 msgr2=0x7ff2180fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 -- 172.21.15.6:0/3145561329 shutdown_connections 2023-11-17T00:05:36.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.648+0000 7ff21ebff700 1 -- 172.21.15.6:0/3145561329 wait complete. 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 Processor -- start 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 -- start start 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff218100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218106040 0x7ff2180ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:36.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180ffa90 con 0x7ff218103720 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180ffc00 con 0x7ff218100e00 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181a0110 con 0x7ff218106040 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff217fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.649+0000 7ff21c99b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff218100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:55360/0 (socket says 172.21.15.6:55360) 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 -- 172.21.15.6:0/1991208463 learned_addr learned my addr 172.21.15.6:0/1991208463 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff21c99b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff218100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33216/0 (socket says 172.21.15.6:33216) 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff21d19c700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218106040 0x7ff2180ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 -- 172.21.15.6:0/1991208463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218106040 msgr2=0x7ff2180ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218106040 0x7ff2180ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 -- 172.21.15.6:0/1991208463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 msgr2=0x7ff218100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff218100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 -- 172.21.15.6:0/1991208463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c0095f0 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff217fff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9a0 tx=0x7ff20400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204009790 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21810bc90 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.650+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff204010430 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.651+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204018680 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.651+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21810bb00 con 0x7ff218103720 2023-11-17T00:05:36.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.652+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2181a0850 con 0x7ff218103720 2023-11-17T00:05:36.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.654+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff2040105a0 con 0x7ff218103720 2023-11-17T00:05:36.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.655+0000 7ff215ffb700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 0x7ff2080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:36.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.655+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff204014070 con 0x7ff218103720 2023-11-17T00:05:36.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.656+0000 7ff21c99b700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 0x7ff2080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:36.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.656+0000 7ff21c99b700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 0x7ff2080a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:36.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.660+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff204051730 con 0x7ff218103720 2023-11-17T00:05:36.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.928+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2181a0e00 con 0x7ff218103720 2023-11-17T00:05:36.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.928+0000 7ff215ffb700 1 -- 172.21.15.6:0/1991208463 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7ff2040961d0 con 0x7ff218103720 2023-11-17T00:05:36.929 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:36.929 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-17T00:05:36.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 msgr2=0x7ff2080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:36.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 0x7ff2080a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 msgr2=0x7ff2180fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9a0 tx=0x7ff20400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 shutdown_connections 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff2080a11d0 0x7ff2080a3680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff218106040 0x7ff2180ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff218100e00 0x7ff218100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 --2- 172.21.15.6:0/1991208463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff218103720 0x7ff2180fedb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 >> 172.21.15.6:0/1991208463 conn(0x7ff2180fa7c0 msgr2=0x7ff2181022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 shutdown_connections 2023-11-17T00:05:36.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:36.931+0000 7ff21ebff700 1 -- 172.21.15.6:0/1991208463 wait complete. 2023-11-17T00:05:36.933 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2023-11-17T00:05:37.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:36 smithi006 ceph-mon[97840]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:37.200 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:36 smithi006 podman[130491]: 2023-11-17T00:05:37.200 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mds[130534]: -- 172.21.15.6:0/1240673243 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557813651520 con 0x557813628800 2023-11-17T00:05:37.200 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-j[130509]: starting mds.j at 2023-11-17T00:05:37.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:36 smithi087 ceph-mon[131182]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:37.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:36 smithi186 ceph-mon[131487]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:37.472 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:37 smithi006 bash[130491]: ed5730b914be23697f96addf7e500f39b11ef4d1fb5c78da3d3c3934ef88fc51 2023-11-17T00:05:37.472 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:05:37 smithi006 systemd[1]: Started Ceph mds.j for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:38.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1991208463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:37 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1991208463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:38.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:37 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1991208463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:38.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:37 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:38.952 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.187+0000 7f47629ff700 1 -- 172.21.15.6:0/2287354829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 msgr2=0x7f475c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.187+0000 7f47629ff700 1 --2- 172.21.15.6:0/2287354829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c072f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f474c009990 tx=0x7f474c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 -- 172.21.15.6:0/2287354829 shutdown_connections 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 --2- 172.21.15.6:0/2287354829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c072f80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 --2- 172.21.15.6:0/2287354829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 0x7f475c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 --2- 172.21.15.6:0/2287354829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 0x7f475c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 -- 172.21.15.6:0/2287354829 >> 172.21.15.6:0/2287354829 conn(0x7f475c0fb8b0 msgr2=0x7f475c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:39.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 -- 172.21.15.6:0/2287354829 shutdown_connections 2023-11-17T00:05:39.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.188+0000 7f47629ff700 1 -- 172.21.15.6:0/2287354829 wait complete. 2023-11-17T00:05:39.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.189+0000 7f47629ff700 1 Processor -- start 2023-11-17T00:05:39.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 -- start start 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 0x7f475c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 0x7f475c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c19fe00 con 0x7f475c074b30 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c19ff40 con 0x7f475c108190 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f47629ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a00d0 con 0x7f475c073550 2023-11-17T00:05:39.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.190+0000 7f475b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43296/0 (socket says 172.21.15.6:43296) 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 -- 172.21.15.6:0/63779178 learned_addr learned my addr 172.21.15.6:0/63779178 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475bfff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 0x7f475c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f4760f9c700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 0x7f475c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 -- 172.21.15.6:0/63779178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 msgr2=0x7f475c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 0x7f475c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 -- 172.21.15.6:0/63779178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 msgr2=0x7f475c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 0x7f475c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 -- 172.21.15.6:0/63779178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0095f0 con 0x7f475c074b30 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f475b7fe700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f475000e970 tx=0x7f475000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750009790 con 0x7f475c074b30 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c10de90 con 0x7f475c074b30 2023-11-17T00:05:39.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475000c7e0 con 0x7f475c074b30 2023-11-17T00:05:39.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.191+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750003c70 con 0x7f475c074b30 2023-11-17T00:05:39.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.192+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c10dd00 con 0x7f475c074b30 2023-11-17T00:05:39.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.193+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c106840 con 0x7f475c074b30 2023-11-17T00:05:39.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.195+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f475000cc50 con 0x7f475c074b30 2023-11-17T00:05:39.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.197+0000 7f47597fa700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 0x7f47440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:39.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.198+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4750014070 con 0x7f475c074b30 2023-11-17T00:05:39.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.198+0000 7f475bfff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 0x7f47440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:39.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.198+0000 7f475bfff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 0x7f47440a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b2e0 tx=0x7f474c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:39.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.200+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4750096350 con 0x7f475c074b30 2023-11-17T00:05:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:38 smithi006 ceph-mon[97840]: Deploying daemon mds.d on smithi006 2023-11-17T00:05:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:38 smithi006 ceph-mon[97840]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:38 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.6:6834/3779928633,v1:172.21.15.6:6835/3779928633] up:boot 2023-11-17T00:05:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:38 smithi006 ceph-mon[97840]: fsmap 3 up:standby 2023-11-17T00:05:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:38 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:05:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:38 smithi087 ceph-mon[131182]: Deploying daemon mds.d on smithi006 2023-11-17T00:05:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:38 smithi087 ceph-mon[131182]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:38 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.6:6834/3779928633,v1:172.21.15.6:6835/3779928633] up:boot 2023-11-17T00:05:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:38 smithi087 ceph-mon[131182]: fsmap 3 up:standby 2023-11-17T00:05:39.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:38 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:05:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:38 smithi186 ceph-mon[131487]: Deploying daemon mds.d on smithi006 2023-11-17T00:05:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:38 smithi186 ceph-mon[131487]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:38 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.6:6834/3779928633,v1:172.21.15.6:6835/3779928633] up:boot 2023-11-17T00:05:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:38 smithi186 ceph-mon[131487]: fsmap 3 up:standby 2023-11-17T00:05:39.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:38 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:05:39.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.480+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475c061700 con 0x7f475c074b30 2023-11-17T00:05:39.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.481+0000 7f47597fa700 1 -- 172.21.15.6:0/63779178 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7f4750003dd0 con 0x7f475c074b30 2023-11-17T00:05:39.481 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:39.481 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-17T00:05:39.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 msgr2=0x7f47440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:39.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 0x7f47440a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b2e0 tx=0x7f474c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 msgr2=0x7f475c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:39.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f475000e970 tx=0x7f475000ec80 comp rx=0 tx=0).stop 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 shutdown_connections 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f47440a1150 0x7f47440a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f475c108190 0x7f475c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f475c073550 0x7f475c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 --2- 172.21.15.6:0/63779178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f475c074b30 0x7f475c1987c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.483+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 >> 172.21.15.6:0/63779178 conn(0x7f475c0fb8b0 msgr2=0x7f475c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.484+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 shutdown_connections 2023-11-17T00:05:39.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:39.484+0000 7f47629ff700 1 -- 172.21.15.6:0/63779178 wait complete. 2023-11-17T00:05:39.486 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 4 2023-11-17T00:05:39.933 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:39 smithi006 systemd[1]: Starting Ceph mds.d for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:39 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/63779178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:40.223 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:40 smithi006 podman[130879]: 2023-11-17T00:05:40.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:39 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/63779178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:40.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:39 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/63779178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:40.722 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:40 smithi006 bash[130879]: 52b1941b78f849c08e429fe47275a04ccba94ed9a82c540fbfa40422b209cfc4 2023-11-17T00:05:40.723 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mds[130919]: -- 172.21.15.6:0/503910916 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564659265520 con 0x56465923d000 2023-11-17T00:05:40.723 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-d[130895]: starting mds.d at 2023-11-17T00:05:40.723 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:05:40 smithi006 systemd[1]: Started Ceph mds.d for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:41.201 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:40 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:40 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:40 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:41.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 -- 172.21.15.6:0/4268380218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 msgr2=0x7fbe641006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 --2- 172.21.15.6:0/4268380218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641006e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009990 tx=0x7fbe58009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 -- 172.21.15.6:0/4268380218 shutdown_connections 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 --2- 172.21.15.6:0/4268380218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe64101e50 0x7fbe64108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 --2- 172.21.15.6:0/4268380218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641006e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 --2- 172.21.15.6:0/4268380218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe641014c0 0x7fbe64101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 -- 172.21.15.6:0/4268380218 >> 172.21.15.6:0/4268380218 conn(0x7fbe640fb8b0 msgr2=0x7fbe640fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 -- 172.21.15.6:0/4268380218 shutdown_connections 2023-11-17T00:05:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.726+0000 7fbe6abff700 1 -- 172.21.15.6:0/4268380218 wait complete. 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.727+0000 7fbe6abff700 1 Processor -- start 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.727+0000 7fbe6abff700 1 -- start start 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe641014c0 0x7fbe641987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe64101e50 0x7fbe6419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419fe20 con 0x7fbe64101e50 2023-11-17T00:05:41.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419ff60 con 0x7fbe641002d0 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6abff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641a00f0 con 0x7fbe641014c0 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43252/0 (socket says 172.21.15.6:43252) 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 -- 172.21.15.6:0/1156884519 learned_addr learned my addr 172.21.15.6:0/1156884519 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6919c700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe64101e50 0x7fbe6419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:41.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe63fff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe641014c0 0x7fbe641987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 -- 172.21.15.6:0/1156884519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe641014c0 msgr2=0x7fbe641987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe641014c0 0x7fbe641987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 -- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe64101e50 msgr2=0x7fbe6419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe64101e50 0x7fbe6419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.728+0000 7fbe6899b700 1 -- 172.21.15.6:0/1156884519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580095f0 con 0x7fbe641002d0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe6899b700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b620 tx=0x7fbe58005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5801d070 con 0x7fbe641002d0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe58021410 con 0x7fbe641002d0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6410bca0 con 0x7fbe641002d0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58017650 con 0x7fbe641002d0 2023-11-17T00:05:41.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.729+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6410bb10 con 0x7fbe641002d0 2023-11-17T00:05:41.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.730+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe64192a00 con 0x7fbe641002d0 2023-11-17T00:05:41.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.732+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbe58021580 con 0x7fbe641002d0 2023-11-17T00:05:41.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.733+0000 7fbe61ffb700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 0x7fbe540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:41.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.733+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe580cd9e0 con 0x7fbe641002d0 2023-11-17T00:05:41.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.734+0000 7fbe63fff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 0x7fbe540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:41.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.734+0000 7fbe63fff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 0x7fbe540a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009c60 tx=0x7fbe50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:41.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.737+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe580975c0 con 0x7fbe641002d0 2023-11-17T00:05:41.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.995+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe6402cfe0 con 0x7fbe641002d0 2023-11-17T00:05:41.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.996+0000 7fbe61ffb700 1 -- 172.21.15.6:0/1156884519 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7fbe58026de0 con 0x7fbe641002d0 2023-11-17T00:05:41.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:41.996 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-17T00:05:41.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 msgr2=0x7fbe540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:41.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 0x7fbe540a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009c60 tx=0x7fbe50009260 comp rx=0 tx=0).stop 2023-11-17T00:05:41.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 msgr2=0x7fbe641982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:41.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b620 tx=0x7fbe58005740 comp rx=0 tx=0).stop 2023-11-17T00:05:41.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 shutdown_connections 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe540a1150 0x7fbe540a3600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe64101e50 0x7fbe6419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe641002d0 0x7fbe641982a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 --2- 172.21.15.6:0/1156884519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe641014c0 0x7fbe641987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 >> 172.21.15.6:0/1156884519 conn(0x7fbe640fb8b0 msgr2=0x7fbe64106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 shutdown_connections 2023-11-17T00:05:42.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:41.998+0000 7fbe6abff700 1 -- 172.21.15.6:0/1156884519 wait complete. 2023-11-17T00:05:42.035 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 5 2023-11-17T00:05:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:41 smithi087 ceph-mon[131182]: Deploying daemon mds.e on smithi087 2023-11-17T00:05:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:41 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.6:6836/3190561786,v1:172.21.15.6:6837/3190561786] up:boot 2023-11-17T00:05:42.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:41 smithi087 ceph-mon[131182]: fsmap 4 up:standby 2023-11-17T00:05:42.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:41 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:05:42.102 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:41 smithi087 systemd[1]: Starting Ceph mds.e for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:41 smithi006 ceph-mon[97840]: Deploying daemon mds.e on smithi087 2023-11-17T00:05:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:41 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.6:6836/3190561786,v1:172.21.15.6:6837/3190561786] up:boot 2023-11-17T00:05:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:41 smithi006 ceph-mon[97840]: fsmap 4 up:standby 2023-11-17T00:05:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:41 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:05:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:41 smithi186 ceph-mon[131487]: Deploying daemon mds.e on smithi087 2023-11-17T00:05:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:41 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.6:6836/3190561786,v1:172.21.15.6:6837/3190561786] up:boot 2023-11-17T00:05:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:41 smithi186 ceph-mon[131487]: fsmap 4 up:standby 2023-11-17T00:05:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:41 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:05:42.454 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:42 smithi087 podman[155771]: 2023-11-17T00:05:42.454 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mds[155812]: -- 172.21.15.87:0/1071491004 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556330e93520 con 0x556330e68800 2023-11-17T00:05:42.454 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-e[155788]: starting mds.e at 2023-11-17T00:05:42.850 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:42 smithi087 bash[155771]: eb76dc3eda173ec42abb65cee951dbfae627cc939a002df5656718df3ba5e681 2023-11-17T00:05:42.851 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:05:42 smithi087 systemd[1]: Started Ceph mds.e for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:43.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1156884519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:43.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:43.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:43.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:42 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1156884519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:42 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1156884519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:43.358 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:42 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:44.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:43 smithi186 ceph-mon[131487]: Deploying daemon mds.f on smithi186 2023-11-17T00:05:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:43 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6834/2088255632,v1:172.21.15.87:6835/2088255632] up:boot 2023-11-17T00:05:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:43 smithi186 ceph-mon[131487]: fsmap 5 up:standby 2023-11-17T00:05:44.091 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:43 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:05:44.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:43 smithi006 ceph-mon[97840]: Deploying daemon mds.f on smithi186 2023-11-17T00:05:44.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:43 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6834/2088255632,v1:172.21.15.87:6835/2088255632] up:boot 2023-11-17T00:05:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:43 smithi006 ceph-mon[97840]: fsmap 5 up:standby 2023-11-17T00:05:44.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:43 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:05:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.250+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3562605721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4074b30 msgr2=0x7f0cd4072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:44.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.250+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3562605721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4074b30 0x7f0cd4072f80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00098f0 tx=0x7f0cd0009c00 comp rx=0 tx=0).stop 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3562605721 shutdown_connections 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3562605721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd410c8a0 0x7f0cd410ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3562605721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd4074b30 0x7f0cd4072f80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3562605721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3562605721 >> 172.21.15.6:0/3562605721 conn(0x7f0cd40fb8b0 msgr2=0x7f0cd40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3562605721 shutdown_connections 2023-11-17T00:05:44.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.251+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3562605721 wait complete. 2023-11-17T00:05:44.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 Processor -- start 2023-11-17T00:05:44.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 -- start start 2023-11-17T00:05:44.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:44.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 0x7f0cd419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:44.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd410c8a0 0x7f0cd41a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a42a0 con 0x7f0cd4074b30 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a43e0 con 0x7f0cd410c8a0 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cdb3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a4570 con 0x7f0cd4073550 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cd919b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cd899a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 0x7f0cd419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cd919b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57824/0 (socket says 172.21.15.6:57824) 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cd919b700 1 -- 172.21.15.6:0/3490157164 learned_addr learned my addr 172.21.15.6:0/3490157164 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.252+0000 7f0cd999c700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd410c8a0 0x7f0cd41a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 -- 172.21.15.6:0/3490157164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd410c8a0 msgr2=0x7f0cd41a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd410c8a0 0x7f0cd41a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 -- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 msgr2=0x7f0cd419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:44.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 0x7f0cd419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 -- 172.21.15.6:0/3490157164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009d00 con 0x7f0cd4073550 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd899a700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 0x7f0cd419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cd919b700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd001c070 con 0x7f0cd4073550 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd000f820 con 0x7f0cd4073550 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0009520 con 0x7f0cd4073550 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.253+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0003a70 con 0x7f0cd4073550 2023-11-17T00:05:44.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.254+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd4112290 con 0x7f0cd4073550 2023-11-17T00:05:44.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.254+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd41023f0 con 0x7f0cd4073550 2023-11-17T00:05:44.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.256+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0cd0020030 con 0x7f0cd4073550 2023-11-17T00:05:44.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.258+0000 7f0cca7fc700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 0x7f0cc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:44.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.258+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0cd00cda40 con 0x7f0cd4073550 2023-11-17T00:05:44.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.258+0000 7f0cd899a700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 0x7f0cc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:44.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.258+0000 7f0cd899a700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 0x7f0cc00a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4074850 tx=0x7f0cc400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:44.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.260+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0cd0097620 con 0x7f0cd4073550 2023-11-17T00:05:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:43 smithi087 ceph-mon[131182]: Deploying daemon mds.f on smithi186 2023-11-17T00:05:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:43 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6834/2088255632,v1:172.21.15.87:6835/2088255632] up:boot 2023-11-17T00:05:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:43 smithi087 ceph-mon[131182]: fsmap 5 up:standby 2023-11-17T00:05:44.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:43 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:05:44.366 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 systemd[1]: Starting Ceph mds.f for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.516+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cd4067af0 con 0x7f0cd4073550 2023-11-17T00:05:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.516+0000 7f0cca7fc700 1 -- 172.21.15.6:0/3490157164 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f0cd009ac40 con 0x7f0cd4073550 2023-11-17T00:05:44.519 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:44.519 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 msgr2=0x7f0cc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 0x7f0cc00a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4074850 tx=0x7f0cc400b410 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 msgr2=0x7f0cd419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009fd0 tx=0x7f0cd000b790 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 shutdown_connections 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0cc00a1150 0x7f0cc00a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd410c8a0 0x7f0cd41a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4073550 0x7f0cd419c720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 --2- 172.21.15.6:0/3490157164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0cd4074b30 0x7f0cd419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 >> 172.21.15.6:0/3490157164 conn(0x7f0cd40fb8b0 msgr2=0x7f0cd40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 shutdown_connections 2023-11-17T00:05:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:44.519+0000 7f0cdb3ff700 1 -- 172.21.15.6:0/3490157164 wait complete. 2023-11-17T00:05:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 6 2023-11-17T00:05:44.756 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 podman[156790]: 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3490157164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:45.117 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-mds[156831]: -- 172.21.15.186:0/1401330110 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b1d161520 con 0x564b1d137000 2023-11-17T00:05:45.117 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-f[156806]: starting mds.f at 2023-11-17T00:05:45.117 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 bash[156790]: 346bf7dfc39e8e9c9dd089c785c4d0ca444b8f2713421d50f02e71e7e03dd382 2023-11-17T00:05:45.117 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:05:44 smithi186 systemd[1]: Started Ceph mds.f for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3490157164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:45.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:44 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3490157164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:44 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:46.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:45 smithi186 ceph-mon[131487]: Deploying daemon mds.c on smithi186 2023-11-17T00:05:46.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:45 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.186:6834/1004385520,v1:172.21.15.186:6835/1004385520] up:boot 2023-11-17T00:05:46.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:45 smithi186 ceph-mon[131487]: fsmap 6 up:standby 2023-11-17T00:05:46.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:45 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:05:46.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:45 smithi006 ceph-mon[97840]: Deploying daemon mds.c on smithi186 2023-11-17T00:05:46.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:45 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.186:6834/1004385520,v1:172.21.15.186:6835/1004385520] up:boot 2023-11-17T00:05:46.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:45 smithi006 ceph-mon[97840]: fsmap 6 up:standby 2023-11-17T00:05:46.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:45 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:05:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:45 smithi087 ceph-mon[131182]: Deploying daemon mds.c on smithi186 2023-11-17T00:05:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:45 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.186:6834/1004385520,v1:172.21.15.186:6835/1004385520] up:boot 2023-11-17T00:05:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:45 smithi087 ceph-mon[131182]: fsmap 6 up:standby 2023-11-17T00:05:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:45 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:05:46.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:46.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 -- 172.21.15.6:0/772206193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 msgr2=0x7f0f90072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 --2- 172.21.15.6:0/772206193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f90072f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009990 tx=0x7f0f80009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 -- 172.21.15.6:0/772206193 shutdown_connections 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 --2- 172.21.15.6:0/772206193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90108190 0x7f0f9010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 --2- 172.21.15.6:0/772206193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f90072f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 --2- 172.21.15.6:0/772206193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90073550 0x7f0f900739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 -- 172.21.15.6:0/772206193 >> 172.21.15.6:0/772206193 conn(0x7f0f900fb8b0 msgr2=0x7f0f900fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 -- 172.21.15.6:0/772206193 shutdown_connections 2023-11-17T00:05:46.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.754+0000 7f0f957ff700 1 -- 172.21.15.6:0/772206193 wait complete. 2023-11-17T00:05:46.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 Processor -- start 2023-11-17T00:05:46.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 -- start start 2023-11-17T00:05:46.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90073550 0x7f0f901982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:46.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 0x7f0f9019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019fe30 con 0x7f0f90108190 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9019ff70 con 0x7f0f90073550 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.755+0000 7f0f957ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901a0100 con 0x7f0f90074b30 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57846/0 (socket says 172.21.15.6:57846) 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f8f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 0x7f0f9019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 -- 172.21.15.6:0/1613950724 learned_addr learned my addr 172.21.15.6:0/1613950724 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f8effd700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90073550 0x7f0f901982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 -- 172.21.15.6:0/1613950724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90073550 msgr2=0x7f0f901982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90073550 0x7f0f901982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 -- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 msgr2=0x7f0f9019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 0x7f0f9019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 -- 172.21.15.6:0/1613950724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800095f0 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f8f7fe700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 0x7f0f9019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f87fff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e970 tx=0x7f0f7800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.756+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78009790 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.757+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9010de90 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.757+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7800c7e0 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.757+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78003c70 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.757+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9010dd00 con 0x7f0f90074b30 2023-11-17T00:05:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.758+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f90106840 con 0x7f0f90074b30 2023-11-17T00:05:46.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.760+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0f7800cc50 con 0x7f0f90074b30 2023-11-17T00:05:46.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.761+0000 7f0f8cff9700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 0x7f0f7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:46.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.761+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f78014070 con 0x7f0f90074b30 2023-11-17T00:05:46.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.761+0000 7f0f8effd700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 0x7f0f7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:46.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.762+0000 7f0f8effd700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 0x7f0f7c0a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b2e0 tx=0x7f0f80005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:46.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:46.763+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f780967f0 con 0x7f0f90074b30 2023-11-17T00:05:46.865 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:46 smithi186 systemd[1]: Starting Ceph mds.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.021+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f90061700 con 0x7f0f90074b30 2023-11-17T00:05:47.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.022+0000 7f0f8cff9700 1 -- 172.21.15.6:0/1613950724 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7f0f78003dd0 con 0x7f0f90074b30 2023-11-17T00:05:47.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:47.024 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-17T00:05:47.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.024+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 msgr2=0x7f0f7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:47.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.024+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 0x7f0f7c0a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b2e0 tx=0x7f0f80005fb0 comp rx=0 tx=0).stop 2023-11-17T00:05:47.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.024+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 msgr2=0x7f0f901987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.024+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800e970 tx=0x7f0f7800ec80 comp rx=0 tx=0).stop 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 shutdown_connections 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0f7c0a1150 0x7f0f7c0a3600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f90108190 0x7f0f9019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f90073550 0x7f0f901982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 --2- 172.21.15.6:0/1613950724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f90074b30 0x7f0f901987f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 >> 172.21.15.6:0/1613950724 conn(0x7f0f900fb8b0 msgr2=0x7f0f900fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 shutdown_connections 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:47.025+0000 7f0f957ff700 1 -- 172.21.15.6:0/1613950724 wait complete. 2023-11-17T00:05:47.027 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2023-11-17T00:05:47.220 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:47 smithi186 podman[157132]: 2023-11-17T00:05:47.220 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:46 smithi186 ceph-mon[131487]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:47.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:46 smithi006 ceph-mon[97840]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:47.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:46 smithi087 ceph-mon[131182]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:47.615 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mds[157172]: -- 172.21.15.186:0/2719171767 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc9e419520 con 0x55bc9e3ef000 2023-11-17T00:05:47.615 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-c[157148]: starting mds.c at 2023-11-17T00:05:47.615 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:47 smithi186 bash[157132]: 4fe53db931cb66213b8dfc201739fcb2682cbb90137cca8fa2ead66d44e338fd 2023-11-17T00:05:47.615 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:05:47 smithi186 systemd[1]: Started Ceph mds.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:48.218 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1613950724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:48.218 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:48.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:48.219 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:47 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1613950724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:48.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:47 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:48.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1613950724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:48.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:47 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:49.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:49.238 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:48 smithi087 systemd[1]: Starting Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:49.238 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:48 smithi087 ceph-mon[131182]: Deploying daemon mds.h on smithi087 2023-11-17T00:05:49.238 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:48 smithi087 ceph-mon[131182]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:49.238 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:48 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.186:6836/3856911296,v1:172.21.15.186:6837/3856911296] up:boot 2023-11-17T00:05:49.238 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:48 smithi087 ceph-mon[131182]: fsmap 7 up:standby 2023-11-17T00:05:49.239 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:48 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 -- 172.21.15.6:0/3235265555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f41014c0 msgr2=0x7f03f4101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/3235265555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f41014c0 0x7f03f4101910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009990 tx=0x7f03e8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 -- 172.21.15.6:0/3235265555 shutdown_connections 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/3235265555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f4101e50 0x7f03f4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/3235265555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41002d0 0x7f03f41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/3235265555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f41014c0 0x7f03f4101910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 -- 172.21.15.6:0/3235265555 >> 172.21.15.6:0/3235265555 conn(0x7f03f40fb8b0 msgr2=0x7f03f40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 -- 172.21.15.6:0/3235265555 shutdown_connections 2023-11-17T00:05:49.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.259+0000 7f03fa3ff700 1 -- 172.21.15.6:0/3235265555 wait complete. 2023-11-17T00:05:49.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 Processor -- start 2023-11-17T00:05:49.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 -- start start 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 0x7f03f41965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f4101e50 0x7f03f419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419dc10 con 0x7f03f4101e50 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.261+0000 7f03fa3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419dd50 con 0x7f03f41014c0 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03fa3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419dee0 con 0x7f03f41002d0 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41886/0 (socket says 172.21.15.6:41886) 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 -- 172.21.15.6:0/1729695064 learned_addr learned my addr 172.21.15.6:0/1729695064 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 0x7f03f41965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:49.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f899c700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f4101e50 0x7f03f419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 -- 172.21.15.6:0/1729695064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 msgr2=0x7f03f41965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 0x7f03f41965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 -- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f4101e50 msgr2=0x7f03f419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f4101e50 0x7f03f419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 -- 172.21.15.6:0/1729695064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80095f0 con 0x7f03f41002d0 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f37fe700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 0x7f03f41965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:49.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.262+0000 7f03f3fff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03e400e970 tx=0x7f03e400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.263+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4009790 con 0x7f03f41002d0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.263+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f410bc80 con 0x7f03f41002d0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.263+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03e400c7e0 con 0x7f03f41002d0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.263+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4003c70 con 0x7f03f41002d0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.263+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f410baf0 con 0x7f03f41002d0 2023-11-17T00:05:49.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.264+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f419e630 con 0x7f03f41002d0 2023-11-17T00:05:49.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.266+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f03e400cc50 con 0x7f03f41002d0 2023-11-17T00:05:49.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.267+0000 7f03f17fa700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 0x7f03dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:49.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.267+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f03e4014070 con 0x7f03f41002d0 2023-11-17T00:05:49.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.267+0000 7f03f37fe700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 0x7f03dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:49.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.268+0000 7f03f37fe700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 0x7f03dc0a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b2e0 tx=0x7f03e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:49.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.269+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f03e4096350 con 0x7f03f41002d0 2023-11-17T00:05:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:48 smithi186 ceph-mon[131487]: Deploying daemon mds.h on smithi087 2023-11-17T00:05:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:48 smithi186 ceph-mon[131487]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:49.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:48 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.186:6836/3856911296,v1:172.21.15.186:6837/3856911296] up:boot 2023-11-17T00:05:49.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:48 smithi186 ceph-mon[131487]: fsmap 7 up:standby 2023-11-17T00:05:49.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:48 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:05:49.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:48 smithi006 ceph-mon[97840]: Deploying daemon mds.h on smithi087 2023-11-17T00:05:49.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:48 smithi006 ceph-mon[97840]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:49.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:48 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.186:6836/3856911296,v1:172.21.15.186:6837/3856911296] up:boot 2023-11-17T00:05:49.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:48 smithi006 ceph-mon[97840]: fsmap 7 up:standby 2023-11-17T00:05:49.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:48 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:05:49.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.521+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03f41071b0 con 0x7f03f41002d0 2023-11-17T00:05:49.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.522+0000 7f03f17fa700 1 -- 172.21.15.6:0/1729695064 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7f03e4003dd0 con 0x7f03f41002d0 2023-11-17T00:05:49.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:49.524 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-17T00:05:49.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 msgr2=0x7f03dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 0x7f03dc0a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b2e0 tx=0x7f03e8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 msgr2=0x7f03f4196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03e400e970 tx=0x7f03e400ec80 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 shutdown_connections 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f03dc0a1150 0x7f03dc0a3600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03f4101e50 0x7f03f419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f41002d0 0x7f03f4196090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 --2- 172.21.15.6:0/1729695064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03f41014c0 0x7f03f41965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 >> 172.21.15.6:0/1729695064 conn(0x7f03f40fb8b0 msgr2=0x7f03f40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 shutdown_connections 2023-11-17T00:05:49.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:49.525+0000 7f03fa3ff700 1 -- 172.21.15.6:0/1729695064 wait complete. 2023-11-17T00:05:49.528 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 8 2023-11-17T00:05:49.533 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:49 smithi087 podman[156112]: 2023-11-17T00:05:49.534 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:49 smithi087 ceph-mds[156152]: -- 172.21.15.87:0/256013460 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d9ac17520 con 0x555d9abecc00 2023-11-17T00:05:49.534 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[156128]: starting mds.h at 2023-11-17T00:05:49.534 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:49 smithi087 bash[156112]: 1fba86f56a149b745089bf9c27e805b5659bb33d72c804971513d7aa409c1125 2023-11-17T00:05:49.850 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:05:49 smithi087 systemd[1]: Started Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:50.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1729695064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:50.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: Deploying daemon mds.i on smithi186 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6836/850260336,v1:172.21.15.87:6837/850260336] up:boot 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: fsmap 8 up:standby 2023-11-17T00:05:50.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:50 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1729695064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: Deploying daemon mds.i on smithi186 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6836/850260336,v1:172.21.15.87:6837/850260336] up:boot 2023-11-17T00:05:50.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: fsmap 8 up:standby 2023-11-17T00:05:50.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:50 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:05:50.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1729695064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: Deploying daemon mds.i on smithi186 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6836/850260336,v1:172.21.15.87:6837/850260336] up:boot 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: fsmap 8 up:standby 2023-11-17T00:05:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:50 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:05:51.242 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 systemd[1]: Starting Ceph mds.i for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:51.546 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:51.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 -- 172.21.15.6:0/865212522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6341002d0 msgr2=0x7fd6341006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:51.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 --2- 172.21.15.6:0/865212522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6341002d0 0x7fd6341006e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd630009990 tx=0x7fd630009ca0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 -- 172.21.15.6:0/865212522 shutdown_connections 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 --2- 172.21.15.6:0/865212522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6341002d0 0x7fd6341006e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 --2- 172.21.15.6:0/865212522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd634101e50 0x7fd634108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 --2- 172.21.15.6:0/865212522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341014c0 0x7fd634101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 -- 172.21.15.6:0/865212522 >> 172.21.15.6:0/865212522 conn(0x7fd6340fb8b0 msgr2=0x7fd6340fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:51.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 -- 172.21.15.6:0/865212522 shutdown_connections 2023-11-17T00:05:51.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.759+0000 7fd63bdff700 1 -- 172.21.15.6:0/865212522 wait complete. 2023-11-17T00:05:51.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.760+0000 7fd63bdff700 1 Processor -- start 2023-11-17T00:05:51.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.760+0000 7fd63bdff700 1 -- start start 2023-11-17T00:05:51.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 0x7fd6341982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:51.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:51.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 0x7fd63419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:51.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419fe20 con 0x7fd634101e50 2023-11-17T00:05:51.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419ff60 con 0x7fd6341002d0 2023-11-17T00:05:51.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6341a00f0 con 0x7fd6341014c0 2023-11-17T00:05:51.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:51.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63a39c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 0x7fd63419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:51.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63939a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41904/0 (socket says 172.21.15.6:41904) 2023-11-17T00:05:51.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd639b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 0x7fd6341982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:51.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.761+0000 7fd63939a700 1 -- 172.21.15.6:0/2450984791 learned_addr learned my addr 172.21.15.6:0/2450984791 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:51.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 -- 172.21.15.6:0/2450984791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 msgr2=0x7fd6341982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:51.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 0x7fd6341982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 -- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 msgr2=0x7fd63419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:51.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 0x7fd63419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:51.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 -- 172.21.15.6:0/2450984791 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009c40 con 0x7fd6341014c0 2023-11-17T00:05:51.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd639b9b700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 0x7fd6341982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:51.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63a39c700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 0x7fd63419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:05:51.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63939a700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e9a0 tx=0x7fd62400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:51.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624009790 con 0x7fd6341014c0 2023-11-17T00:05:51.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62400ce80 con 0x7fd6341014c0 2023-11-17T00:05:51.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6300095f0 con 0x7fd6341014c0 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624018680 con 0x7fd6341014c0 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.762+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63410bca0 con 0x7fd6341014c0 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.763+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634192a00 con 0x7fd6341014c0 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.765+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd624010430 con 0x7fd6341014c0 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.766+0000 7fd62affd700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 0x7fd6200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:51.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.767+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd624014070 con 0x7fd6341014c0 2023-11-17T00:05:51.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.767+0000 7fd639b9b700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 0x7fd6200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:51.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.767+0000 7fd639b9b700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 0x7fd6200a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b2e0 tx=0x7fd630005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:51.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:51.770+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd624051730 con 0x7fd6341014c0 2023-11-17T00:05:51.846 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 podman[157478]: 2023-11-17T00:05:52.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.033+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63402cfe0 con 0x7fd6341014c0 2023-11-17T00:05:52.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.034+0000 7fd62affd700 1 -- 172.21.15.6:0/2450984791 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6556 (secure 0 0 0) 0x7fd624099cd0 con 0x7fd6341014c0 2023-11-17T00:05:52.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:52.038 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-17T00:05:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 msgr2=0x7fd6200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 0x7fd6200a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b2e0 tx=0x7fd630005fb0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 msgr2=0x7fd6341987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e9a0 tx=0x7fd62400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 shutdown_connections 2023-11-17T00:05:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd6200a1150 0x7fd6200a3600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd634101e50 0x7fd63419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6341002d0 0x7fd6341982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 --2- 172.21.15.6:0/2450984791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6341014c0 0x7fd6341987e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 >> 172.21.15.6:0/2450984791 conn(0x7fd6340fb8b0 msgr2=0x7fd634106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 shutdown_connections 2023-11-17T00:05:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:52.037+0000 7fd63bdff700 1 -- 172.21.15.6:0/2450984791 wait complete. 2023-11-17T00:05:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-17T00:05:52.115 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 ceph-mds[157518]: -- 172.21.15.186:0/1038259121 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588e9acf520 con 0x5588e9aa4c00 2023-11-17T00:05:52.115 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: starting mds.i at 2023-11-17T00:05:52.115 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 bash[157478]: d814852f934aa232bd39cfc1d800327b54063c14a5dc15acfd2ecccfb6e3c780 2023-11-17T00:05:52.116 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:05:51 smithi186 systemd[1]: Started Ceph mds.i for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:53.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:53.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2450984791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:53.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.186:6838/143156596,v1:172.21.15.186:6839/143156596] up:boot 2023-11-17T00:05:53.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: fsmap 9 up:standby 2023-11-17T00:05:53.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:52 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2450984791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:53.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.186:6838/143156596,v1:172.21.15.186:6839/143156596] up:boot 2023-11-17T00:05:53.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: fsmap 9 up:standby 2023-11-17T00:05:53.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:52 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:05:53.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:53.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2450984791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.186:6838/143156596,v1:172.21.15.186:6839/143156596] up:boot 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: fsmap 9 up:standby 2023-11-17T00:05:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:52 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:05:54.060 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:54.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:53 smithi006 ceph-mon[97840]: Deploying daemon mds.g on smithi006 2023-11-17T00:05:54.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:53 smithi087 ceph-mon[131182]: Deploying daemon mds.g on smithi006 2023-11-17T00:05:54.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:53 smithi186 ceph-mon[131487]: Deploying daemon mds.g on smithi006 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 -- 172.21.15.6:0/2618503612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 msgr2=0x7fef681083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 --2- 172.21.15.6:0/2618503612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef681083e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef6000b0a0 tx=0x7fef6000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 -- 172.21.15.6:0/2618503612 shutdown_connections 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 --2- 172.21.15.6:0/2618503612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef681083e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 --2- 172.21.15.6:0/2618503612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 0x7fef68103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 --2- 172.21.15.6:0/2618503612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef68105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 -- 172.21.15.6:0/2618503612 >> 172.21.15.6:0/2618503612 conn(0x7fef680fa7c0 msgr2=0x7fef680fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 -- 172.21.15.6:0/2618503612 shutdown_connections 2023-11-17T00:05:54.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.422+0000 7fef701ff700 1 -- 172.21.15.6:0/2618503612 wait complete. 2023-11-17T00:05:54.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 Processor -- start 2023-11-17T00:05:54.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 -- start start 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 0x7fef68194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef68191730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68191ed0 con 0x7fef68105f80 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68192040 con 0x7fef68103660 2023-11-17T00:05:54.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.424+0000 7fef701ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68192200 con 0x7fef68100d40 2023-11-17T00:05:54.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.425+0000 7fef6d79a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:54.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.425+0000 7fef6d79a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:48590/0 (socket says 172.21.15.6:48590) 2023-11-17T00:05:54.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.425+0000 7fef6d79a700 1 -- 172.21.15.6:0/316329306 learned_addr learned my addr 172.21.15.6:0/316329306 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:54.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.425+0000 7fef6e79c700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef68191730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.425+0000 7fef6df9b700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 0x7fef68194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 -- 172.21.15.6:0/316329306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 msgr2=0x7fef68194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 0x7fef68194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 -- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 msgr2=0x7fef68191730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef68191730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 -- 172.21.15.6:0/316329306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60009d00 con 0x7fef68103660 2023-11-17T00:05:54.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef6d79a700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef5800e9a0 tx=0x7fef5800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58009790 con 0x7fef68103660 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef58010430 con 0x7fef68103660 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6810bcd0 con 0x7fef68103660 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.426+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58018680 con 0x7fef68103660 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.427+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6810bb40 con 0x7fef68103660 2023-11-17T00:05:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.428+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef68107af0 con 0x7fef68103660 2023-11-17T00:05:54.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.429+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fef580105a0 con 0x7fef68103660 2023-11-17T00:05:54.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.431+0000 7fef5effd700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 0x7fef540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:54.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.431+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fef58014070 con 0x7fef68103660 2023-11-17T00:05:54.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.431+0000 7fef6df9b700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 0x7fef540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:54.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.432+0000 7fef6df9b700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 0x7fef540a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef64005950 tx=0x7fef6400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:54.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.435+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef58051730 con 0x7fef68103660 2023-11-17T00:05:54.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.710+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef68061700 con 0x7fef68103660 2023-11-17T00:05:54.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.711+0000 7fef5effd700 1 -- 172.21.15.6:0/316329306 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7315 (secure 0 0 0) 0x7fef58096210 con 0x7fef68103660 2023-11-17T00:05:54.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:54.713 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-17T00:05:54.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 msgr2=0x7fef540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 0x7fef540a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef64005950 tx=0x7fef6400b410 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 msgr2=0x7fef681950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef5800e9a0 tx=0x7fef5800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 shutdown_connections 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fef540a1150 0x7fef540a3600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef68105f80 0x7fef68191730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef68100d40 0x7fef68194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 --2- 172.21.15.6:0/316329306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef68103660 0x7fef681950e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 >> 172.21.15.6:0/316329306 conn(0x7fef680fa7c0 msgr2=0x7fef680ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 shutdown_connections 2023-11-17T00:05:54.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:54.714+0000 7fef701ff700 1 -- 172.21.15.6:0/316329306 wait complete. 2023-11-17T00:05:54.717 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 10 2023-11-17T00:05:54.722 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:54 smithi006 systemd[1]: Starting Ceph mds.g for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:55.042 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:54 smithi006 ceph-mon[97840]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:55.042 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:54 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/316329306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:55.043 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:54 smithi006 podman[131488]: 2023-11-17T00:05:55.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:54 smithi087 ceph-mon[131182]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:55.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:54 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/316329306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:57.698 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:05:57.870 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:55 smithi006 ceph-mds[131530]: -- 172.21.15.6:0/724402572 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55acf01c9520 con 0x55acf01a0800 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:55 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-g[131506]: starting mds.g at 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:55 smithi006 bash[131488]: 4212766931ac74c5996d00e6c554c030f7f20b036ec78059a4a975d83a4a6f60 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:05:55 smithi006 systemd[1]: Started Ceph mds.g for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:57.871 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: Deploying daemon mds.a on smithi006 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.6:6838/3324637103,v1:172.21.15.6:6839/3324637103] up:boot 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: fsmap 10 up:standby 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:56 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:57.872 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:57.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:57.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: Deploying daemon mds.a on smithi006 2023-11-17T00:05:57.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:58.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.6:6838/3324637103,v1:172.21.15.6:6839/3324637103] up:boot 2023-11-17T00:05:58.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: fsmap 10 up:standby 2023-11-17T00:05:58.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:56 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:05:58.052 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:54 smithi186 ceph-mon[131487]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:58.098 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:54 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/316329306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:58.098 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: Deploying daemon mds.a on smithi006 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.6:6838/3324637103,v1:172.21.15.6:6839/3324637103] up:boot 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: fsmap 10 up:standby 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:56 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:05:58.099 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:57 smithi006 systemd[1]: Starting Ceph mds.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 -- 172.21.15.6:0/3954838553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec1014c0 msgr2=0x7f01ec101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3954838553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec1014c0 0x7f01ec101910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01e00098f0 tx=0x7f01e0009c00 comp rx=0 tx=0).stop 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 -- 172.21.15.6:0/3954838553 shutdown_connections 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3954838553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec101e50 0x7f01ec108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3954838553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1002d0 0x7f01ec1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3954838553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec1014c0 0x7f01ec101910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 -- 172.21.15.6:0/3954838553 >> 172.21.15.6:0/3954838553 conn(0x7f01ec0fb8b0 msgr2=0x7f01ec0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:58.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 -- 172.21.15.6:0/3954838553 shutdown_connections 2023-11-17T00:05:58.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.317+0000 7f01f1fff700 1 -- 172.21.15.6:0/3954838553 wait complete. 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.318+0000 7f01f1fff700 1 Processor -- start 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.318+0000 7f01f1fff700 1 -- start start 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1014c0 0x7f01ec1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 0x7f01ec19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19dc20 con 0x7f01ec101e50 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19dd60 con 0x7f01ec1002d0 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01f1fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19def0 con 0x7f01ec1014c0 2023-11-17T00:05:58.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 0x7f01ec19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01ebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 0x7f01ec19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35658/0 (socket says 172.21.15.6:35658) 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:48634/0 (socket says 172.21.15.6:48634) 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01ebfff700 1 -- 172.21.15.6:0/3522144 learned_addr learned my addr 172.21.15.6:0/3522144 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eaffd700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1014c0 0x7f01ec1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 -- 172.21.15.6:0/3522144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1014c0 msgr2=0x7f01ec1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1014c0 0x7f01ec1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 -- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 msgr2=0x7f01ec19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 0x7f01ec19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.319+0000 7f01eb7fe700 1 -- 172.21.15.6:0/3522144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc0095f0 con 0x7f01ec1002d0 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01eb7fe700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005fd0 tx=0x7f01dc00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00cbd0 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01dc00cd30 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e0009520 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc018710 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.320+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec10bc80 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.321+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec10be10 con 0x7f01ec1002d0 2023-11-17T00:05:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.323+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f01dc010980 con 0x7f01ec1002d0 2023-11-17T00:05:58.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.324+0000 7f01e8ff9700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:05:58.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.324+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01dc014070 con 0x7f01ec1002d0 2023-11-17T00:05:58.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.324+0000 7f01eaffd700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:05:58.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.325+0000 7f01eaffd700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 0x7f01d40a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e0005e90 tx=0x7f01e0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:05:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.328+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f01dc0964d0 con 0x7f01ec1002d0 2023-11-17T00:05:58.333 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:58 smithi006 podman[131833]: 2023-11-17T00:05:58.333 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:58 smithi006 bash[131833]: 11f75eebf1217722852ea5074098610cac0462229525182979ec8a6e99e273a7 2023-11-17T00:05:58.333 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mds[131896]: -- 172.21.15.6:0/1031254499 <== mon.0 v2:172.21.15.6:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de4ae73520 con 0x55de4ae4a800 2023-11-17T00:05:58.333 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-a[131857]: starting mds.a at 2023-11-17T00:05:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.591+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01ec1071d0 con 0x7f01ec1002d0 2023-11-17T00:05:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.592+0000 7f01e8ff9700 1 -- 172.21.15.6:0/3522144 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8070 (secure 0 0 0) 0x7f01dc099af0 con 0x7f01ec1002d0 2023-11-17T00:05:58.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:05:58.593 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 msgr2=0x7f01d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 0x7f01d40a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e0005e90 tx=0x7f01e0019040 comp rx=0 tx=0).stop 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 msgr2=0x7f01ec1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005fd0 tx=0x7f01dc00ee70 comp rx=0 tx=0).stop 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 shutdown_connections 2023-11-17T00:05:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f01d40a1150 0x7f01d40a3600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec101e50 0x7f01ec19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01ec1002d0 0x7f01ec1960a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 --2- 172.21.15.6:0/3522144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec1014c0 0x7f01ec1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 >> 172.21.15.6:0/3522144 conn(0x7f01ec0fb8b0 msgr2=0x7f01ec0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 shutdown_connections 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:05:58.594+0000 7f01f1fff700 1 -- 172.21.15.6:0/3522144 wait complete. 2023-11-17T00:05:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 11 2023-11-17T00:05:58.613 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:05:58 smithi006 systemd[1]: Started Ceph mds.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:58 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3522144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:58 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3522144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:05:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:05:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:05:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-17T00:05:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-17T00:05:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:05:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:58 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3522144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:00.101 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:05:59 smithi087 systemd[1]: Starting Ceph mds.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:06:08.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:59 smithi087 ceph-mon[131182]: Deploying daemon mds.b on smithi087 2023-11-17T00:06:08.053 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:59 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.6:6840/1535787123,v1:172.21.15.6:6841/1535787123] up:boot 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:59 smithi087 ceph-mon[131182]: fsmap 11 up:standby 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:05:59 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:00 smithi087 podman[156452]: 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mds[156492]: -- 172.21.15.87:0/2604004498 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b25957520 con 0x561b2592cc00 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: starting mds.b at 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:00 smithi087 bash[156452]: 725d4b84a7982a6506a707198b4d8653d7f723fe070e75144faa8e3aebba37dc 2023-11-17T00:06:08.054 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:00 smithi087 systemd[1]: Started Ceph mds.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:06:08.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.055 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:00 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:01 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:boot 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:01 smithi087 ceph-mon[131182]: fsmap 12 up:standby 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:02 smithi087 ceph-mon[131182]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.056 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:04 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:04 smithi087 ceph-mon[131182]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:05 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:06:08.057 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.155 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:08.155 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.155 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:06 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:06:08.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:59 smithi006 ceph-mon[97840]: Deploying daemon mds.b on smithi087 2023-11-17T00:06:08.158 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:59 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.6:6840/1535787123,v1:172.21.15.6:6841/1535787123] up:boot 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:59 smithi006 ceph-mon[97840]: fsmap 11 up:standby 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:05:59 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:00 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:01 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:boot 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:01 smithi006 ceph-mon[97840]: fsmap 12 up:standby 2023-11-17T00:06:08.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:02 smithi006 ceph-mon[97840]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:04 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:04 smithi006 ceph-mon[97840]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:05 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:06 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:59 smithi186 ceph-mon[131487]: Deploying daemon mds.b on smithi087 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:59 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.6:6840/1535787123,v1:172.21.15.6:6841/1535787123] up:boot 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:59 smithi186 ceph-mon[131487]: fsmap 11 up:standby 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:05:59 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:00 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:01 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:boot 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:01 smithi186 ceph-mon[131487]: fsmap 12 up:standby 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:02 smithi186 ceph-mon[131487]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:04 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:04 smithi186 ceph-mon[131487]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:05 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:08.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.165 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:06 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 -- 172.21.15.6:0/1557134389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170100340 msgr2=0x7f9170100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 --2- 172.21.15.6:0/1557134389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170100340 0x7f9170100750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f91640098f0 tx=0x7f9164009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 -- 172.21.15.6:0/1557134389 shutdown_connections 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 --2- 172.21.15.6:0/1557134389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101ec0 0x7f9170108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 --2- 172.21.15.6:0/1557134389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170100340 0x7f9170100750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 --2- 172.21.15.6:0/1557134389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170101530 0x7f9170101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 -- 172.21.15.6:0/1557134389 >> 172.21.15.6:0/1557134389 conn(0x7f91700fb940 msgr2=0x7f91700fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 -- 172.21.15.6:0/1557134389 shutdown_connections 2023-11-17T00:06:08.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.375+0000 7f9176bff700 1 -- 172.21.15.6:0/1557134389 wait complete. 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.376+0000 7f9176bff700 1 Processor -- start 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.376+0000 7f9176bff700 1 -- start start 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101530 0x7f9170196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 0x7f917019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019dc70 con 0x7f9170100340 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019ddb0 con 0x7f9170101ec0 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f9176bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917019df40 con 0x7f9170101530 2023-11-17T00:06:08.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53152/0 (socket says 172.21.15.6:53152) 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 -- 172.21.15.6:0/3080555072 learned_addr learned my addr 172.21.15.6:0/3080555072 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917519c700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 0x7f917019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f916ffff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101530 0x7f9170196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 -- 172.21.15.6:0/3080555072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101530 msgr2=0x7f9170196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101530 0x7f9170196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 -- 172.21.15.6:0/3080555072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 msgr2=0x7f917019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 0x7f917019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917499b700 1 -- 172.21.15.6:0/3080555072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164009520 con 0x7f9170100340 2023-11-17T00:06:08.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.377+0000 7f917519c700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 0x7f917019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:08.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f917499b700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9164005e90 tx=0x7f916400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:08.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916401c070 con 0x7f9170100340 2023-11-17T00:06:08.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916400f7f0 con 0x7f9170100340 2023-11-17T00:06:08.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164003990 con 0x7f9170100340 2023-11-17T00:06:08.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917010bca0 con 0x7f9170100340 2023-11-17T00:06:08.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.378+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917010bb10 con 0x7f9170100340 2023-11-17T00:06:08.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.379+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917019e4e0 con 0x7f9170100340 2023-11-17T00:06:08.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.381+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9164020030 con 0x7f9170100340 2023-11-17T00:06:08.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.383+0000 7f916dffb700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 0x7f91600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.383+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91640cd830 con 0x7f9170100340 2023-11-17T00:06:08.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.384+0000 7f916ffff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 0x7f91600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.384+0000 7f916ffff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 0x7f91600a3680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f915c009c60 tx=0x7f915c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:08.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.385+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9164097410 con 0x7f9170100340 2023-11-17T00:06:08.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.655+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9170107270 con 0x7f9170100340 2023-11-17T00:06:08.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.656+0000 7f916dffb700 1 -- 172.21.15.6:0/3080555072 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9580 (secure 0 0 0) 0x7f916409aa30 con 0x7f9170100340 2023-11-17T00:06:08.658 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:08.658 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-17T00:06:08.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 msgr2=0x7f91600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 0x7f91600a3680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f915c009c60 tx=0x7f915c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:08.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 msgr2=0x7f91701960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9164005e90 tx=0x7f916400b730 comp rx=0 tx=0).stop 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 shutdown_connections 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91600a11d0 0x7f91600a3680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9170100340 0x7f91701960f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9170101ec0 0x7f917019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 --2- 172.21.15.6:0/3080555072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9170101530 0x7f9170196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 >> 172.21.15.6:0/3080555072 conn(0x7f91700fb940 msgr2=0x7f91700fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:08.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 shutdown_connections 2023-11-17T00:06:08.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.659+0000 7f9176bff700 1 -- 172.21.15.6:0/3080555072 wait complete. 2023-11-17T00:06:08.662 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 13 2023-11-17T00:06:08.682 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 -- 172.21.15.6:0/1382616655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 msgr2=0x7f7990072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 --2- 172.21.15.6:0/1382616655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f7990072f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7980009990 tx=0x7f7980009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 -- 172.21.15.6:0/1382616655 shutdown_connections 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 --2- 172.21.15.6:0/1382616655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 0x7f799010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 --2- 172.21.15.6:0/1382616655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f7990072f80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 --2- 172.21.15.6:0/1382616655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f79900739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 -- 172.21.15.6:0/1382616655 >> 172.21.15.6:0/1382616655 conn(0x7f79900fb8b0 msgr2=0x7f79900fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:08.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 -- 172.21.15.6:0/1382616655 shutdown_connections 2023-11-17T00:06:08.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.897+0000 7f7994fff700 1 -- 172.21.15.6:0/1382616655 wait complete. 2023-11-17T00:06:08.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.898+0000 7f7994fff700 1 Processor -- start 2023-11-17T00:06:08.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.898+0000 7f7994fff700 1 -- start start 2023-11-17T00:06:08.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.898+0000 7f7994fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.898+0000 7f7994fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f799019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f7994fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 0x7f79901a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f7994fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a4260 con 0x7f7990073550 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f7994fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a43a0 con 0x7f7990074b30 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f7994fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901a4530 con 0x7f799010c8a0 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46734/0 (socket says 172.21.15.6:46734) 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 -- 172.21.15.6:0/1159345172 learned_addr learned my addr 172.21.15.6:0/1159345172 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798dd9b700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f799019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798ed9d700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 0x7f79901a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 -- 172.21.15.6:0/1159345172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 msgr2=0x7f79901a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 0x7f79901a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 -- 172.21.15.6:0/1159345172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 msgr2=0x7f799019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f799019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.899+0000 7f798e59c700 1 -- 172.21.15.6:0/1159345172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800095f0 con 0x7f7990073550 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f798e59c700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7980004710 tx=0x7f7980009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798001d070 con 0x7f7990073550 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7980021410 con 0x7f7990073550 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7990112290 con 0x7f7990073550 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980017650 con 0x7f7990073550 2023-11-17T00:06:08.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.900+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990112100 con 0x7f7990073550 2023-11-17T00:06:08.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.901+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79901023f0 con 0x7f7990073550 2023-11-17T00:06:08.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.904+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7980021580 con 0x7f7990073550 2023-11-17T00:06:08.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.905+0000 7f797f7fe700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 0x7f79780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:08.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.905+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79800cd8f0 con 0x7f7990073550 2023-11-17T00:06:08.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.905+0000 7f798dd9b700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 0x7f79780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:08.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.906+0000 7f798dd9b700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 0x7f79780a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7990074850 tx=0x7f7984009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:08.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:08.909+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f79800974d0 con 0x7f7990073550 2023-11-17T00:06:08.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3080555072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:08 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:09.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3080555072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:09.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:08 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3080555072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:09.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:08 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.133+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f799002cfe0 con 0x7f7990073550 2023-11-17T00:06:09.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.134+0000 7f797f7fe700 1 -- 172.21.15.6:0/1159345172 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7f798009aaf0 con 0x7f7990073550 2023-11-17T00:06:09.137 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:09.137 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":80,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:05:21.318020+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:09.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 msgr2=0x7f79780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 0x7f79780a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7990074850 tx=0x7f7984009260 comp rx=0 tx=0).stop 2023-11-17T00:06:09.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 msgr2=0x7f799019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7980004710 tx=0x7f7980009580 comp rx=0 tx=0).stop 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 shutdown_connections 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f79780a1150 0x7f79780a3600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7990073550 0x7f799019c6e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f799010c8a0 0x7f79901a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 --2- 172.21.15.6:0/1159345172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7990074b30 0x7f799019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 >> 172.21.15.6:0/1159345172 conn(0x7f79900fb8b0 msgr2=0x7f79900fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:09.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 shutdown_connections 2023-11-17T00:06:09.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.137+0000 7f7994fff700 1 -- 172.21.15.6:0/1159345172 wait complete. 2023-11-17T00:06:09.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-17T00:06:09.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:06:09.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 -- 172.21.15.6:0/1305778826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a01002a0 msgr2=0x7fa5a01006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/1305778826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a01002a0 0x7fa5a01006b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa590009990 tx=0x7fa590009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 -- 172.21.15.6:0/1305778826 shutdown_connections 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/1305778826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a01002a0 0x7fa5a01006b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/1305778826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a0101e20 0x7fa5a01086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/1305778826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101490 0x7fa5a01018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 -- 172.21.15.6:0/1305778826 >> 172.21.15.6:0/1305778826 conn(0x7fa5a00fb880 msgr2=0x7fa5a00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 -- 172.21.15.6:0/1305778826 shutdown_connections 2023-11-17T00:06:09.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.374+0000 7fa5a5bff700 1 -- 172.21.15.6:0/1305778826 wait complete. 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 Processor -- start 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 -- start start 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 0x7fa5a0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0073bc0 con 0x7fa5a0101490 2023-11-17T00:06:09.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0073d30 con 0x7fa5a01002a0 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa5a5bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a019ff60 con 0x7fa5a0101e20 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa59effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa59f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa59f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57706/0 (socket says 172.21.15.6:57706) 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.375+0000 7fa59effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46764/0 (socket says 172.21.15.6:46764) 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 -- 172.21.15.6:0/51478598 learned_addr learned my addr 172.21.15.6:0/51478598 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59ffff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 0x7fa5a0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 -- 172.21.15.6:0/51478598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 msgr2=0x7fa5a0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 0x7fa5a0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 -- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 msgr2=0x7fa5a0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 -- 172.21.15.6:0/51478598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900095f0 con 0x7fa5a01002a0 2023-11-17T00:06:09.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59ffff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 0x7fa5a0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59effd700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59f7fe700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5900059d0 tx=0x7fa590005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.376+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59001d070 con 0x7fa5a01002a0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.377+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a010bc00 con 0x7fa5a01002a0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.377+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa590004020 con 0x7fa5a01002a0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.377+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590022d00 con 0x7fa5a01002a0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.377+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a010ba70 con 0x7fa5a01002a0 2023-11-17T00:06:09.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.378+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a0107180 con 0x7fa5a01002a0 2023-11-17T00:06:09.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.380+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa590022470 con 0x7fa5a01002a0 2023-11-17T00:06:09.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.381+0000 7fa59cff9700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 0x7fa5880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.381+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5900cdff0 con 0x7fa5a01002a0 2023-11-17T00:06:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.381+0000 7fa59effd700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 0x7fa5880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.382+0000 7fa59effd700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 0x7fa5880a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa59400afd0 tx=0x7fa59400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:09.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.383+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa590097c10 con 0x7fa5a01002a0 2023-11-17T00:06:09.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.599+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5a0061ad0 con 0x7fa5a01002a0 2023-11-17T00:06:09.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.600+0000 7fa59cff9700 1 -- 172.21.15.6:0/51478598 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa590022e60 con 0x7fa5a01002a0 2023-11-17T00:06:09.601 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 msgr2=0x7fa5880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 0x7fa5880a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa59400afd0 tx=0x7fa59400a380 comp rx=0 tx=0).stop 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 msgr2=0x7fa5a00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5900059d0 tx=0x7fa590005740 comp rx=0 tx=0).stop 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 shutdown_connections 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa5880a1150 0x7fa5880a3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5a0101e20 0x7fa5a0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5a01002a0 0x7fa5a00748a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 --2- 172.21.15.6:0/51478598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5a0101490 0x7fa5a0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 >> 172.21.15.6:0/51478598 conn(0x7fa5a00fb880 msgr2=0x7fa5a00fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:09.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 shutdown_connections 2023-11-17T00:06:09.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.602+0000 7fa5a5bff700 1 -- 172.21.15.6:0/51478598 wait complete. 2023-11-17T00:06:09.623 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-17T00:06:09.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2536044511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01014c0 msgr2=0x7fb0a0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2536044511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01014c0 0x7fb0a0101910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb094009990 tx=0x7fb094009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2536044511 shutdown_connections 2023-11-17T00:06:09.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2536044511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a0101e50 0x7fb0a0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2536044511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a01002d0 0x7fb0a01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2536044511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01014c0 0x7fb0a0101910 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.834+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2536044511 >> 172.21.15.6:0/2536044511 conn(0x7fb0a00fb8b0 msgr2=0x7fb0a00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:09.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.835+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2536044511 shutdown_connections 2023-11-17T00:06:09.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.835+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2536044511 wait complete. 2023-11-17T00:06:09.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.836+0000 7fb0a5bff700 1 Processor -- start 2023-11-17T00:06:09.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.836+0000 7fb0a5bff700 1 -- start start 2023-11-17T00:06:09.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a01014c0 0x7fb0a0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 0x7fb0a0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0073bc0 con 0x7fb0a01002d0 2023-11-17T00:06:09.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0073d30 con 0x7fb0a01014c0 2023-11-17T00:06:09.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb0a5bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a01a0090 con 0x7fb0a0101e50 2023-11-17T00:06:09.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46782/0 (socket says 172.21.15.6:46782) 2023-11-17T00:06:09.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 -- 172.21.15.6:0/2337014198 learned_addr learned my addr 172.21.15.6:0/2337014198 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:09.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09effd700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a01014c0 0x7fb0a0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09ffff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 0x7fb0a0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 -- 172.21.15.6:0/2337014198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 msgr2=0x7fb0a0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 0x7fb0a0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 -- 172.21.15.6:0/2337014198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a01014c0 msgr2=0x7fb0a0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a01014c0 0x7fb0a0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09f7fe700 1 -- 172.21.15.6:0/2337014198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940095f0 con 0x7fb0a01002d0 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.837+0000 7fb09ffff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 0x7fb0a0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:06:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb09f7fe700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e970 tx=0x7fb09000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090009790 con 0x7fb0a01002d0 2023-11-17T00:06:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09000eec0 con 0x7fb0a01002d0 2023-11-17T00:06:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a010bc80 con 0x7fb0a01002d0 2023-11-17T00:06:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090003cf0 con 0x7fb0a01002d0 2023-11-17T00:06:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.838+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a010baf0 con 0x7fb0a01002d0 2023-11-17T00:06:09.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.839+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a01a0830 con 0x7fb0a01002d0 2023-11-17T00:06:09.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.841+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb09000ccd0 con 0x7fb0a01002d0 2023-11-17T00:06:09.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.843+0000 7fb09cff9700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 0x7fb0880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:09.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.843+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb090014070 con 0x7fb0a01002d0 2023-11-17T00:06:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.843+0000 7fb09effd700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 0x7fb0880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.844+0000 7fb09effd700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 0x7fb0880a3680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb09400b2e0 tx=0x7fb094005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:09.847+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb090096210 con 0x7fb0a01002d0 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1159345172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/51478598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:09.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:09 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.059+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb0a0107160 con 0x7fb0a01002d0 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.059+0000 7fb09cff9700 1 -- 172.21.15.6:0/2337014198 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb090099830 con 0x7fb0a01002d0 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:06:10.061 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2023-11-17T00:06:10.062 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2023-11-17T00:06:10.062 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:06:10.062 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 msgr2=0x7fb0880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 0x7fb0880a3680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb09400b2e0 tx=0x7fb094005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 msgr2=0x7fb0a00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e970 tx=0x7fb09000ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 shutdown_connections 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0880a11d0 0x7fb0880a3680 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a01002d0 0x7fb0a00748a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a0101e50 0x7fb0a0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 --2- 172.21.15.6:0/2337014198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a01014c0 0x7fb0a0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 >> 172.21.15.6:0/2337014198 conn(0x7fb0a00fb8b0 msgr2=0x7fb0a00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:10.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 shutdown_connections 2023-11-17T00:06:10.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.062+0000 7fb0a5bff700 1 -- 172.21.15.6:0/2337014198 wait complete. 2023-11-17T00:06:10.083 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-17T00:06:10.083 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1159345172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/51478598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:10.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:09 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1159345172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:10.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/51478598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:06:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:06:10.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:09 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/3023944098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 msgr2=0x7f3c981006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/3023944098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c981006a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009990 tx=0x7f3c88009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/3023944098 shutdown_connections 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/3023944098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c981006a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/3023944098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 0x7f3c981086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/3023944098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 0x7f3c981018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/3023944098 >> 172.21.15.6:0/3023944098 conn(0x7f3c980fb870 msgr2=0x7f3c980fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/3023944098 shutdown_connections 2023-11-17T00:06:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.290+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/3023944098 wait complete. 2023-11-17T00:06:10.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.291+0000 7f3c9c9ff700 1 Processor -- start 2023-11-17T00:06:10.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 -- start start 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 0x7f3c98072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 0x7f3c98073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98073bc0 con 0x7f3c98100290 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98073d30 con 0x7f3c98101480 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9c9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c981a0090 con 0x7f3c98101e10 2023-11-17T00:06:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46808/0 (socket says 172.21.15.6:46808) 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.292+0000 7f3c9659c700 1 -- 172.21.15.6:0/2008814623 learned_addr learned my addr 172.21.15.6:0/2008814623 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c8ffff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 0x7f3c98072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c96d9d700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 0x7f3c98073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 -- 172.21.15.6:0/2008814623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 msgr2=0x7f3c98073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 0x7f3c98073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 -- 172.21.15.6:0/2008814623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 msgr2=0x7f3c98072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 0x7f3c98072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 -- 172.21.15.6:0/2008814623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c880095f0 con 0x7f3c98100290 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9659c700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800b620 tx=0x7f3c88005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8801d070 con 0x7f3c98100290 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c88021410 con 0x7f3c98100290 2023-11-17T00:06:10.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9810bc40 con 0x7f3c98100290 2023-11-17T00:06:10.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.293+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88017650 con 0x7f3c98100290 2023-11-17T00:06:10.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.294+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9810bab0 con 0x7f3c98100290 2023-11-17T00:06:10.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.295+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c981a0690 con 0x7f3c98100290 2023-11-17T00:06:10.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.297+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3c88021580 con 0x7f3c98100290 2023-11-17T00:06:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.298+0000 7f3c8f7fe700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 0x7f3c840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.298+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3c880cd8e0 con 0x7f3c98100290 2023-11-17T00:06:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.298+0000 7f3c8ffff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 0x7f3c840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.299+0000 7f3c8ffff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 0x7f3c840a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009c60 tx=0x7f3c80009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:10.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.302+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3c880974c0 con 0x7f3c98100290 2023-11-17T00:06:10.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:10.512+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3c98107280 con 0x7f3c98100290 2023-11-17T00:06:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.078+0000 7f3c8f7fe700 1 -- 172.21.15.6:0/2008814623 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3c8809aae0 con 0x7f3c98100290 2023-11-17T00:06:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.081+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 msgr2=0x7f3c840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.081+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 0x7f3c840a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009c60 tx=0x7f3c80009260 comp rx=0 tx=0).stop 2023-11-17T00:06:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.081+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 msgr2=0x7f3c980748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.081+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800b620 tx=0x7f3c88005740 comp rx=0 tx=0).stop 2023-11-17T00:06:11.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 shutdown_connections 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c840a11d0 0x7f3c840a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c98100290 0x7f3c980748a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c98101e10 0x7f3c98073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 --2- 172.21.15.6:0/2008814623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c98101480 0x7f3c98072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 >> 172.21.15.6:0/2008814623 conn(0x7f3c980fb870 msgr2=0x7f3c980fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.082+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 shutdown_connections 2023-11-17T00:06:11.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.083+0000 7f3c9c9ff700 1 -- 172.21.15.6:0/2008814623 wait complete. 2023-11-17T00:06:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2023-11-17T00:06:11.106 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-17T00:06:11.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 -- 172.21.15.6:0/301824393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90100290 msgr2=0x7fce901006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:11.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 --2- 172.21.15.6:0/301824393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90100290 0x7fce901006a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce800098f0 tx=0x7fce80009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:11.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 -- 172.21.15.6:0/301824393 shutdown_connections 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 --2- 172.21.15.6:0/301824393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 0x7fce901086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 --2- 172.21.15.6:0/301824393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90100290 0x7fce901006a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 --2- 172.21.15.6:0/301824393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90101480 0x7fce901018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.314+0000 7fce963ff700 1 -- 172.21.15.6:0/301824393 >> 172.21.15.6:0/301824393 conn(0x7fce900fb870 msgr2=0x7fce900fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.315+0000 7fce963ff700 1 -- 172.21.15.6:0/301824393 shutdown_connections 2023-11-17T00:06:11.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.315+0000 7fce963ff700 1 -- 172.21.15.6:0/301824393 wait complete. 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 Processor -- start 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 -- start start 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 0x7fce9019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9019fda0 con 0x7fce90100290 2023-11-17T00:06:11.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9019fee0 con 0x7fce90101480 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce963ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce901a0070 con 0x7fce90101e10 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46826/0 (socket says 172.21.15.6:46826) 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57756/0 (socket says 172.21.15.6:57756) 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce8ffff700 1 -- 172.21.15.6:0/4072093161 learned_addr learned my addr 172.21.15.6:0/4072093161 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.316+0000 7fce9499c700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 0x7fce9019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:11.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 -- 172.21.15.6:0/4072093161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 msgr2=0x7fce9019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:11.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 0x7fce9019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 -- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 msgr2=0x7fce90198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:11.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 -- 172.21.15.6:0/4072093161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80009520 con 0x7fce90101480 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8ffff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8f7fe700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce8400e9c0 tx=0x7fce8400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8400c9c0 con 0x7fce90101480 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9010bbf0 con 0x7fce90101480 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce84010430 con 0x7fce90101480 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.317+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84018680 con 0x7fce90101480 2023-11-17T00:06:11.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.318+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9010ba60 con 0x7fce90101480 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.319+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90067a50 con 0x7fce90101480 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.321+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fce84010860 con 0x7fce90101480 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.322+0000 7fce8d7fa700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 0x7fce780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.322+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fce84014070 con 0x7fce90101480 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.322+0000 7fce8ffff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 0x7fce780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:11.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.323+0000 7fce8ffff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 0x7fce780a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c010 tx=0x7fce80019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:11.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.324+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fce84096070 con 0x7fce90101480 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:11.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:11 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2337014198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-17T00:06:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:06:11.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:11.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:11.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:11 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2337014198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-17T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:11 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2337014198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-17T00:06:11.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:11.533+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fce901a0b80 con 0x7fce90101480 2023-11-17T00:06:12.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.084+0000 7fce8d7fa700 1 -- 172.21.15.6:0/4072093161 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7fce84099690 con 0x7fce90101480 2023-11-17T00:06:12.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 msgr2=0x7fce780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 0x7fce780a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c010 tx=0x7fce80019040 comp rx=0 tx=0).stop 2023-11-17T00:06:12.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 msgr2=0x7fce90198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce8400e9c0 tx=0x7fce8400ecd0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 shutdown_connections 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fce780a1150 0x7fce780a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce90100290 0x7fce90198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce90101e10 0x7fce9019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 --2- 172.21.15.6:0/4072093161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce90101480 0x7fce90198760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.089+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 >> 172.21.15.6:0/4072093161 conn(0x7fce900fb870 msgr2=0x7fce900fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:12.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.090+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 shutdown_connections 2023-11-17T00:06:12.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.090+0000 7fce963ff700 1 -- 172.21.15.6:0/4072093161 wait complete. 2023-11-17T00:06:12.094 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2023-11-17T00:06:12.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-17T00:06:12.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 -- 172.21.15.6:0/2646485320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 msgr2=0x7fe2bc1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/2646485320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc1086d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400b0a0 tx=0x7fe2b400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 -- 172.21.15.6:0/2646485320 shutdown_connections 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/2646485320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc1086d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/2646485320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 0x7fe2bc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/2646485320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 -- 172.21.15.6:0/2646485320 >> 172.21.15.6:0/2646485320 conn(0x7fe2bc0fb870 msgr2=0x7fe2bc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 -- 172.21.15.6:0/2646485320 shutdown_connections 2023-11-17T00:06:12.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.328+0000 7fe2c3fff700 1 -- 172.21.15.6:0/2646485320 wait complete. 2023-11-17T00:06:12.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 Processor -- start 2023-11-17T00:06:12.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 -- start start 2023-11-17T00:06:12.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 0x7fe2bc1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19dc40 con 0x7fe2bc101480 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19dd80 con 0x7fe2bc101e10 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19df10 con 0x7fe2bc100290 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.329+0000 7fe2c159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57782/0 (socket says 172.21.15.6:57782) 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 -- 172.21.15.6:0/1269951123 learned_addr learned my addr 172.21.15.6:0/1269951123 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c1d9b700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 0x7fe2bc1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 -- 172.21.15.6:0/1269951123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 msgr2=0x7fe2bc1960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 0x7fe2bc1960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 -- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 msgr2=0x7fe2bc196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 -- 172.21.15.6:0/1269951123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009d00 con 0x7fe2bc101e10 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c159a700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c259c700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d010 tx=0x7fe2b400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400e040 con 0x7fe2bc101e10 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc10bc70 con 0x7fe2bc101e10 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b4003e10 con 0x7fe2bc101e10 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.330+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40052a0 con 0x7fe2bc101e10 2023-11-17T00:06:12.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.331+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc10bae0 con 0x7fe2bc101e10 2023-11-17T00:06:12.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.332+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc1077d0 con 0x7fe2bc101e10 2023-11-17T00:06:12.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.334+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe2b401d480 con 0x7fe2bc101e10 2023-11-17T00:06:12.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.335+0000 7fe2b2ffd700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 0x7fe2a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.335+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fe2b40ce150 con 0x7fe2bc101e10 2023-11-17T00:06:12.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.335+0000 7fe2c1d9b700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 0x7fe2a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.336+0000 7fe2c1d9b700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 0x7fe2a80a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc1012e0 tx=0x7fe2b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:12.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.337+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe2b4097a40 con 0x7fe2bc101e10 2023-11-17T00:06:12.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:12 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-17T00:06:12.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:12 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-17T00:06:12.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:12 smithi087 ceph-mon[131182]: osdmap e81: 12 total, 12 up, 12 in 2023-11-17T00:06:12.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:12 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4072093161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:12 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-17T00:06:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:12 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-17T00:06:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:12 smithi186 ceph-mon[131487]: osdmap e81: 12 total, 12 up, 12 in 2023-11-17T00:06:12.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:12 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4072093161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:12.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:12 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-17T00:06:12.475 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:12 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2008814623' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-17T00:06:12.476 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:12 smithi006 ceph-mon[97840]: osdmap e81: 12 total, 12 up, 12 in 2023-11-17T00:06:12.477 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:12 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4072093161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:12.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.595+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe2bc061700 con 0x7fe2bc101e10 2023-11-17T00:06:12.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.604+0000 7fe2b2ffd700 1 -- 172.21.15.6:0/1269951123 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe2b409b060 con 0x7fe2bc101e10 2023-11-17T00:06:12.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 msgr2=0x7fe2a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 0x7fe2a80a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc1012e0 tx=0x7fe2b8009260 comp rx=0 tx=0).stop 2023-11-17T00:06:12.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 msgr2=0x7fe2bc19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d010 tx=0x7fe2b400bc80 comp rx=0 tx=0).stop 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 shutdown_connections 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe2a80a1150 0x7fe2a80a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2bc101e10 0x7fe2bc19d670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2bc100290 0x7fe2bc1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 --2- 172.21.15.6:0/1269951123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe2bc101480 0x7fe2bc196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.609+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 >> 172.21.15.6:0/1269951123 conn(0x7fe2bc0fb870 msgr2=0x7fe2bc104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:12.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.610+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 shutdown_connections 2023-11-17T00:06:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.612+0000 7fe2c3fff700 1 -- 172.21.15.6:0/1269951123 wait complete. 2023-11-17T00:06:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 2 and data pool 3 2023-11-17T00:06:12.644 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-17T00:06:12.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 -- 172.21.15.6:0/949822722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8100290 msgr2=0x7fb0a81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 --2- 172.21.15.6:0/949822722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8100290 0x7fb0a81006a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009990 tx=0x7fb0a4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 -- 172.21.15.6:0/949822722 shutdown_connections 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 --2- 172.21.15.6:0/949822722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8100290 0x7fb0a81006a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 --2- 172.21.15.6:0/949822722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 --2- 172.21.15.6:0/949822722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8101480 0x7fb0a81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 -- 172.21.15.6:0/949822722 >> 172.21.15.6:0/949822722 conn(0x7fb0a80fb870 msgr2=0x7fb0a80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 -- 172.21.15.6:0/949822722 shutdown_connections 2023-11-17T00:06:12.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.861+0000 7fb0affff700 1 -- 172.21.15.6:0/949822722 wait complete. 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.862+0000 7fb0affff700 1 Processor -- start 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.862+0000 7fb0affff700 1 -- start start 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.862+0000 7fb0affff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8100290 0x7fb0a8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.862+0000 7fb0affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0affff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0affff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819fda0 con 0x7fb0a8101480 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0affff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819fee0 con 0x7fb0a8101e10 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0affff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a81a0070 con 0x7fb0a8100290 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46868/0 (socket says 172.21.15.6:46868) 2023-11-17T00:06:12.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 -- 172.21.15.6:0/1183035088 learned_addr learned my addr 172.21.15.6:0/1183035088 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0add9b700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8100290 0x7fb0a8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ae59c700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 -- 172.21.15.6:0/1183035088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8100290 msgr2=0x7fb0a8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8100290 0x7fb0a8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 -- 172.21.15.6:0/1183035088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 msgr2=0x7fb0a819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 -- 172.21.15.6:0/1183035088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0009c40 con 0x7fb0a8101480 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ae59c700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.863+0000 7fb0ad59a700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9a0 tx=0x7fb09800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.864+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098009790 con 0x7fb0a8101480 2023-11-17T00:06:12.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.864+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09800cbd0 con 0x7fb0a8101480 2023-11-17T00:06:12.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.864+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a40095f0 con 0x7fb0a8101480 2023-11-17T00:06:12.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.864+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098018680 con 0x7fb0a8101480 2023-11-17T00:06:12.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.864+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a810bc40 con 0x7fb0a8101480 2023-11-17T00:06:12.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.865+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a8067a50 con 0x7fb0a8101480 2023-11-17T00:06:12.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.867+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb09800cd40 con 0x7fb0a8101480 2023-11-17T00:06:12.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.868+0000 7fb09effd700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 0x7fb0940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:12.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.868+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb098014070 con 0x7fb0a8101480 2023-11-17T00:06:12.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.868+0000 7fb0add9b700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 0x7fb0940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:12.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.869+0000 7fb0add9b700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 0x7fb0940a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b2e0 tx=0x7fb0a4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:12.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:12.870+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb0980964f0 con 0x7fb0a8101480 2023-11-17T00:06:13.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.092+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb0a81a0bd0 con 0x7fb0a8101480 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: pgmap v286: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: osdmap e82: 12 total, 12 up, 12 in 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1269951123' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: osdmap e83: 12 total, 12 up, 12 in 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-17T00:06:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-17T00:06:13.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:13 smithi087 ceph-mon[131182]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: pgmap v286: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: osdmap e82: 12 total, 12 up, 12 in 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1269951123' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-17T00:06:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: osdmap e83: 12 total, 12 up, 12 in 2023-11-17T00:06:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-17T00:06:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-17T00:06:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:13 smithi186 ceph-mon[131487]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-17T00:06:13.478 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: pgmap v286: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: osdmap e82: 12 total, 12 up, 12 in 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1269951123' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-17T00:06:13.490 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: osdmap e83: 12 total, 12 up, 12 in 2023-11-17T00:06:13.491 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-17T00:06:13.491 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-17T00:06:13.491 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:13 smithi006 ceph-mon[97840]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-17T00:06:13.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.607+0000 7fb09effd700 1 -- 172.21.15.6:0/1183035088 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb0980260e0 con 0x7fb0a8101480 2023-11-17T00:06:13.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 msgr2=0x7fb0940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 0x7fb0940a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b2e0 tx=0x7fb0a4005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 msgr2=0x7fb0a8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9a0 tx=0x7fb09800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 shutdown_connections 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0940a11d0 0x7fb0940a3680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a8101e10 0x7fb0a819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0a8100290 0x7fb0a8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 --2- 172.21.15.6:0/1183035088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0a8101480 0x7fb0a8198760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 >> 172.21.15.6:0/1183035088 conn(0x7fb0a80fb870 msgr2=0x7fb0a80fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:13.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 shutdown_connections 2023-11-17T00:06:13.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.610+0000 7fb0affff700 1 -- 172.21.15.6:0/1183035088 wait complete. 2023-11-17T00:06:13.635 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 -- 172.21.15.6:0/3188914785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 msgr2=0x7fe748072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/3188914785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748072f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe738009990 tx=0x7fe738009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 -- 172.21.15.6:0/3188914785 shutdown_connections 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/3188914785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748072f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/3188914785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 0x7fe74810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/3188914785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe7480739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 -- 172.21.15.6:0/3188914785 >> 172.21.15.6:0/3188914785 conn(0x7fe7480fb890 msgr2=0x7fe7480fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 -- 172.21.15.6:0/3188914785 shutdown_connections 2023-11-17T00:06:13.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.852+0000 7fe74c9ff700 1 -- 172.21.15.6:0/3188914785 wait complete. 2023-11-17T00:06:13.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 Processor -- start 2023-11-17T00:06:13.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 -- start start 2023-11-17T00:06:13.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe748198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:13.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 0x7fe74819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819fd80 con 0x7fe748074b30 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819fec0 con 0x7fe748073550 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.853+0000 7fe74c9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7481a0050 con 0x7fe748108170 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe74659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe748198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe74659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe748198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57824/0 (socket says 172.21.15.6:57824) 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe74659c700 1 -- 172.21.15.6:0/2755635686 learned_addr learned my addr 172.21.15.6:0/2755635686 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe746d9d700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 0x7fe74819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 -- 172.21.15.6:0/2755635686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 msgr2=0x7fe74819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 0x7fe74819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 -- 172.21.15.6:0/2755635686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 msgr2=0x7fe748198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe748198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 -- 172.21.15.6:0/2755635686 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009520 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73ffff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748198740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe73000e9c0 tx=0x7fe73000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73000cb70 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe73000ee60 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7380095f0 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.854+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730018680 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.855+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74810de70 con 0x7fe748074b30 2023-11-17T00:06:13.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.856+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74810e000 con 0x7fe748074b30 2023-11-17T00:06:13.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.858+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe730010430 con 0x7fe748074b30 2023-11-17T00:06:13.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.859+0000 7fe73f7fe700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 0x7fe7340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:13.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.859+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fe730014070 con 0x7fe748074b30 2023-11-17T00:06:13.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.859+0000 7fe74659c700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 0x7fe7340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:13.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.860+0000 7fe74659c700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 0x7fe7340a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b2e0 tx=0x7fe738005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:13.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:13.863+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe730096440 con 0x7fe748074b30 2023-11-17T00:06:14.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:14.077+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe7480617a0 con 0x7fe748074b30 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:active 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: osdmap e84: 12 total, 12 up, 12 in 2023-11-17T00:06:14.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:14 smithi087 ceph-mon[131182]: pgmap v290: 97 pgs: 12 creating+peering, 27 active+clean, 58 unknown; 578 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:active 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: osdmap e84: 12 total, 12 up, 12 in 2023-11-17T00:06:14.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:14 smithi186 ceph-mon[131487]: pgmap v290: 97 pgs: 12 creating+peering, 27 active+clean, 58 unknown; 578 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-11-17T00:06:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-17T00:06:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] up:active 2023-11-17T00:06:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:14.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1183035088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-17T00:06:14.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: osdmap e84: 12 total, 12 up, 12 in 2023-11-17T00:06:14.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:14 smithi006 ceph-mon[97840]: pgmap v290: 97 pgs: 12 creating+peering, 27 active+clean, 58 unknown; 578 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-11-17T00:06:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-17T00:06:15.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-17T00:06:15.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-17T00:06:16.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.073+0000 7fe73f7fe700 1 -- 172.21.15.6:0/2755635686 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7fe73000ce60 con 0x7fe748074b30 2023-11-17T00:06:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.078+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 msgr2=0x7fe7340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.078+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 0x7fe7340a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b2e0 tx=0x7fe738005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.078+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 msgr2=0x7fe748198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:16.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.078+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748198740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe73000e9c0 tx=0x7fe73000ecd0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.079+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 shutdown_connections 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.079+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe7340a1150 0x7fe7340a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.079+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe748108170 0x7fe74819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.079+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe748073550 0x7fe748198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.079+0000 7fe74c9ff700 1 --2- 172.21.15.6:0/2755635686 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe748074b30 0x7fe748198740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.080+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 >> 172.21.15.6:0/2755635686 conn(0x7fe7480fb890 msgr2=0x7fe7480fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:16.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.080+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 shutdown_connections 2023-11-17T00:06:16.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.080+0000 7fe74c9ff700 1 -- 172.21.15.6:0/2755635686 wait complete. 2023-11-17T00:06:16.083 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' created 2023-11-17T00:06:16.101 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:06:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:06:15.818+0000 7f5b0609a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-17T00:06:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:16 smithi087 ceph-mon[131182]: osdmap e85: 12 total, 12 up, 12 in 2023-11-17T00:06:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:16 smithi087 ceph-mon[131182]: pgmap v292: 97 pgs: 12 creating+peering, 66 active+clean, 19 unknown; 578 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-11-17T00:06:16.103 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 -- 172.21.15.6:0/442153876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 msgr2=0x7f2fbc1005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/442153876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc1005c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0098f0 tx=0x7f2fac009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 -- 172.21.15.6:0/442153876 shutdown_connections 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/442153876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc101d30 0x7f2fbc1085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/442153876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc1005c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/442153876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 -- 172.21.15.6:0/442153876 >> 172.21.15.6:0/442153876 conn(0x7f2fbc0fb790 msgr2=0x7f2fbc0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 -- 172.21.15.6:0/442153876 shutdown_connections 2023-11-17T00:06:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.312+0000 7f2fc0bff700 1 -- 172.21.15.6:0/442153876 wait complete. 2023-11-17T00:06:16.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 Processor -- start 2023-11-17T00:06:16.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 -- start start 2023-11-17T00:06:16.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:16.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 0x7f2fbc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc19fdd0 con 0x7f2fbc1013a0 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc19ff10 con 0x7f2fbc1001b0 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.313+0000 7f2fc0bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1a00a0 con 0x7f2fbc101d30 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:16.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46888/0 (socket says 172.21.15.6:46888) 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 -- 172.21.15.6:0/240289637 learned_addr learned my addr 172.21.15.6:0/240289637 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fba59c700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fbad9d700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 0x7f2fbc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 -- 172.21.15.6:0/240289637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 msgr2=0x7f2fbc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 0x7f2fbc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 -- 172.21.15.6:0/240289637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 msgr2=0x7f2fbc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 -- 172.21.15.6:0/240289637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009520 con 0x7f2fbc1013a0 2023-11-17T00:06:16.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fbad9d700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 0x7f2fbc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fba59c700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb3fff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e9c0 tx=0x7f2fa400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.314+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400cbd0 con 0x7f2fbc1013a0 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.315+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa400eeb0 con 0x7f2fbc1013a0 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.315+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc10bb70 con 0x7f2fbc1013a0 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.315+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4005580 con 0x7f2fbc1013a0 2023-11-17T00:06:16.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.315+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc10b9e0 con 0x7f2fbc1013a0 2023-11-17T00:06:16.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.316+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc0679e0 con 0x7f2fbc1013a0 2023-11-17T00:06:16.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.318+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2fa4004750 con 0x7f2fbc1013a0 2023-11-17T00:06:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.320+0000 7f2fb37fe700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 0x7f2fa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.320+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f2fa4014070 con 0x7f2fbc1013a0 2023-11-17T00:06:16.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.320+0000 7f2fba59c700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 0x7f2fa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:16.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.320+0000 7f2fba59c700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 0x7f2fa80a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009fd0 tx=0x7f2fac019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:16.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.322+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2fa4096750 con 0x7f2fbc1013a0 2023-11-17T00:06:16.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:16 smithi186 ceph-mon[131487]: osdmap e85: 12 total, 12 up, 12 in 2023-11-17T00:06:16.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:16 smithi186 ceph-mon[131487]: pgmap v292: 97 pgs: 12 creating+peering, 66 active+clean, 19 unknown; 578 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-11-17T00:06:16.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:16 smithi006 ceph-mon[97840]: osdmap e85: 12 total, 12 up, 12 in 2023-11-17T00:06:16.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:16 smithi006 ceph-mon[97840]: pgmap v292: 97 pgs: 12 creating+peering, 66 active+clean, 19 unknown; 578 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-11-17T00:06:16.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:16.531+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f2fbc1a0f70 con 0x7f2fbc1013a0 2023-11-17T00:06:17.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.075+0000 7f2fb37fe700 1 -- 172.21.15.6:0/240289637 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2fa4099d70 con 0x7f2fbc1013a0 2023-11-17T00:06:17.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.081+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 msgr2=0x7f2fa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.081+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 0x7f2fa80a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009fd0 tx=0x7f2fac019040 comp rx=0 tx=0).stop 2023-11-17T00:06:17.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.081+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 msgr2=0x7f2fbc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.081+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e9c0 tx=0x7f2fa400ecd0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 shutdown_connections 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2fa80a1150 0x7f2fa80a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fbc101d30 0x7f2fbc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fbc1001b0 0x7f2fbc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 --2- 172.21.15.6:0/240289637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fbc1013a0 0x7f2fbc198790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.082+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 >> 172.21.15.6:0/240289637 conn(0x7f2fbc0fb790 msgr2=0x7f2fbc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:17.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.084+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 shutdown_connections 2023-11-17T00:06:17.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.084+0000 7f2fc0bff700 1 -- 172.21.15.6:0/240289637 wait complete. 2023-11-17T00:06:17.091 INFO:teuthology.orchestra.run.smithi006.stderr:set pool 4 allow_ec_overwrites to true 2023-11-17T00:06:17.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-17T00:06:17.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.327+0000 7fd9addff700 1 -- 172.21.15.6:0/1390681823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8100290 msgr2=0x7fd9a81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.327+0000 7fd9addff700 1 --2- 172.21.15.6:0/1390681823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8100290 0x7fd9a81006a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd998009990 tx=0x7fd998009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 -- 172.21.15.6:0/1390681823 shutdown_connections 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 --2- 172.21.15.6:0/1390681823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 0x7fd9a81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 --2- 172.21.15.6:0/1390681823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8100290 0x7fd9a81006a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 --2- 172.21.15.6:0/1390681823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8101480 0x7fd9a81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 -- 172.21.15.6:0/1390681823 >> 172.21.15.6:0/1390681823 conn(0x7fd9a80fb870 msgr2=0x7fd9a80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 -- 172.21.15.6:0/1390681823 shutdown_connections 2023-11-17T00:06:17.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.328+0000 7fd9addff700 1 -- 172.21.15.6:0/1390681823 wait complete. 2023-11-17T00:06:17.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.329+0000 7fd9addff700 1 Processor -- start 2023-11-17T00:06:17.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.329+0000 7fd9addff700 1 -- start start 2023-11-17T00:06:17.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8101480 0x7fd9a8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 0x7fd9a8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8073bc0 con 0x7fd9a8100290 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8073d30 con 0x7fd9a8101480 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9addff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a0060 con 0x7fd9a8101e10 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46904/0 (socket says 172.21.15.6:46904) 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 -- 172.21.15.6:0/3221843889 learned_addr learned my addr 172.21.15.6:0/3221843889 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a6ffd700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8101480 0x7fd9a8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a7fff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 0x7fd9a8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 -- 172.21.15.6:0/3221843889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 msgr2=0x7fd9a8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 0x7fd9a8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 -- 172.21.15.6:0/3221843889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8101480 msgr2=0x7fd9a8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8101480 0x7fd9a8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.330+0000 7fd9a77fe700 1 -- 172.21.15.6:0/3221843889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009520 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9a77fe700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd998004730 tx=0x7fd998004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99801d070 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd998021410 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9980095f0 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998017650 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.331+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a810bc40 con 0x7fd9a8100290 2023-11-17T00:06:17.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.332+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8067a50 con 0x7fd9a8100290 2023-11-17T00:06:17.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.334+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd998021580 con 0x7fd9a8100290 2023-11-17T00:06:17.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.335+0000 7fd9a4ff9700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 0x7fd9900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.336+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fd9980cdea0 con 0x7fd9a8100290 2023-11-17T00:06:17.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.336+0000 7fd9a6ffd700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 0x7fd9900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.336+0000 7fd9a6ffd700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 0x7fd9900a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009df0 tx=0x7fd99c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:17.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.339+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd998097450 con 0x7fd9a8100290 2023-11-17T00:06:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-17T00:06:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:17 smithi087 ceph-mon[131182]: osdmap e86: 12 total, 12 up, 12 in 2023-11-17T00:06:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-17T00:06:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-17T00:06:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:17 smithi186 ceph-mon[131487]: osdmap e86: 12 total, 12 up, 12 in 2023-11-17T00:06:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-17T00:06:17.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2755635686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-17T00:06:17.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:17 smithi006 ceph-mon[97840]: osdmap e86: 12 total, 12 up, 12 in 2023-11-17T00:06:17.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-17T00:06:17.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.599+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fd9a8107240 con 0x7fd9a8100290 2023-11-17T00:06:17.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.605+0000 7fd9a4ff9700 1 -- 172.21.15.6:0/3221843889 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7fd99809aa70 con 0x7fd9a8100290 2023-11-17T00:06:17.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 msgr2=0x7fd9900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 0x7fd9900a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009df0 tx=0x7fd99c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 msgr2=0x7fd9a80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd998004730 tx=0x7fd998004760 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 shutdown_connections 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd9900a11d0 0x7fd9900a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9a8100290 0x7fd9a80748a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9a8101e10 0x7fd9a8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 --2- 172.21.15.6:0/3221843889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9a8101480 0x7fd9a8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 >> 172.21.15.6:0/3221843889 conn(0x7fd9a80fb870 msgr2=0x7fd9a80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 shutdown_connections 2023-11-17T00:06:17.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.608+0000 7fd9addff700 1 -- 172.21.15.6:0/3221843889 wait complete. 2023-11-17T00:06:17.611 INFO:teuthology.orchestra.run.smithi006.stderr:added data pool 4 to fsmap 2023-11-17T00:06:17.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:06:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 -- 172.21.15.6:0/2898945470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1014c0 msgr2=0x7f8b0c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 --2- 172.21.15.6:0/2898945470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c101910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009990 tx=0x7f8b00009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 -- 172.21.15.6:0/2898945470 shutdown_connections 2023-11-17T00:06:17.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 --2- 172.21.15.6:0/2898945470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 0x7f8b0c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 --2- 172.21.15.6:0/2898945470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 --2- 172.21.15.6:0/2898945470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c101910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 -- 172.21.15.6:0/2898945470 >> 172.21.15.6:0/2898945470 conn(0x7f8b0c0fb8b0 msgr2=0x7f8b0c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 -- 172.21.15.6:0/2898945470 shutdown_connections 2023-11-17T00:06:17.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.831+0000 7f8b127ff700 1 -- 172.21.15.6:0/2898945470 wait complete. 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.832+0000 7f8b127ff700 1 Processor -- start 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.832+0000 7f8b127ff700 1 -- start start 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 0x7f8b0c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c073bc0 con 0x7f8b0c1002d0 2023-11-17T00:06:17.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c073d30 con 0x7f8b0c1014c0 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b127ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a00b0 con 0x7f8b0c101e50 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46910/0 (socket says 172.21.15.6:46910) 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 -- 172.21.15.6:0/803097437 learned_addr learned my addr 172.21.15.6:0/803097437 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0b7fe700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b10d9c700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 0x7f8b0c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 -- 172.21.15.6:0/803097437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 msgr2=0x7f8b0c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 0x7f8b0c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 -- 172.21.15.6:0/803097437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1014c0 msgr2=0x7f8b0c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.833+0000 7f8b0bfff700 1 -- 172.21.15.6:0/803097437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000095f0 con 0x7f8b0c1002d0 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b0bfff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e970 tx=0x7f8afc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:17.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc009790 con 0x7f8b0c1002d0 2023-11-17T00:06:17.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8afc00c7e0 con 0x7f8b0c1002d0 2023-11-17T00:06:17.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c10bc80 con 0x7f8b0c1002d0 2023-11-17T00:06:17.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc003c70 con 0x7f8b0c1002d0 2023-11-17T00:06:17.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.834+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c10baf0 con 0x7f8b0c1002d0 2023-11-17T00:06:17.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.835+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c1a0800 con 0x7f8b0c1002d0 2023-11-17T00:06:17.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.837+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8afc00cc50 con 0x7f8b0c1002d0 2023-11-17T00:06:17.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.839+0000 7f8b097fa700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 0x7f8af40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:17.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.839+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8afc014070 con 0x7f8b0c1002d0 2023-11-17T00:06:17.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.839+0000 7f8b0b7fe700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 0x7f8af40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:17.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.840+0000 7f8b0b7fe700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 0x7f8af40a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000b2e0 tx=0x7f8b00005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:17.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:17.843+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8afc096210 con 0x7f8b0c1002d0 2023-11-17T00:06:18.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.099+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b0c1071d0 con 0x7f8b0c1002d0 2023-11-17T00:06:18.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.100+0000 7f8b097fa700 1 -- 172.21.15.6:0/803097437 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10726 (secure 0 0 0) 0x7f8afc003dd0 con 0x7f8b0c1002d0 2023-11-17T00:06:18.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:18.101 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:17.601623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-17T00:06:18.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 msgr2=0x7f8af40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 0x7f8af40a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000b2e0 tx=0x7f8b00005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 msgr2=0x7f8b0c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e970 tx=0x7f8afc00ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:18.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 shutdown_connections 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8af40a1150 0x7f8af40a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b0c1002d0 0x7f8b0c0748a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b0c101e50 0x7f8b0c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 --2- 172.21.15.6:0/803097437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b0c1014c0 0x7f8b0c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.102+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 >> 172.21.15.6:0/803097437 conn(0x7f8b0c0fb8b0 msgr2=0x7f8b0c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.103+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 shutdown_connections 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.103+0000 7f8b127ff700 1 -- 172.21.15.6:0/803097437 wait complete. 2023-11-17T00:06:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-17T00:06:18.124 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.327+0000 7f07033ff700 1 -- 172.21.15.6:0/2807772886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1014c0 msgr2=0x7f06fc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.327+0000 7f07033ff700 1 --2- 172.21.15.6:0/2807772886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1014c0 0x7f06fc101910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009990 tx=0x7f06ec009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 -- 172.21.15.6:0/2807772886 shutdown_connections 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 --2- 172.21.15.6:0/2807772886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 0x7f06fc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 --2- 172.21.15.6:0/2807772886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1002d0 0x7f06fc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 --2- 172.21.15.6:0/2807772886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1014c0 0x7f06fc101910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 -- 172.21.15.6:0/2807772886 >> 172.21.15.6:0/2807772886 conn(0x7f06fc0fb8b0 msgr2=0x7f06fc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:18.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 -- 172.21.15.6:0/2807772886 shutdown_connections 2023-11-17T00:06:18.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.328+0000 7f07033ff700 1 -- 172.21.15.6:0/2807772886 wait complete. 2023-11-17T00:06:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 Processor -- start 2023-11-17T00:06:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 -- start start 2023-11-17T00:06:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 0x7f06fc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc19fde0 con 0x7f06fc1002d0 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc19ff20 con 0x7f06fc1014c0 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f07033ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1a00b0 con 0x7f06fc101e50 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f070119b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f070099a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.329+0000 7f070119b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46934/0 (socket says 172.21.15.6:46934) 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57876/0 (socket says 172.21.15.6:57876) 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070119b700 1 -- 172.21.15.6:0/2642989192 learned_addr learned my addr 172.21.15.6:0/2642989192 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070199c700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 0x7f06fc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 -- 172.21.15.6:0/2642989192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 msgr2=0x7f06fc19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 0x7f06fc19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 -- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 msgr2=0x7f06fc198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 -- 172.21.15.6:0/2642989192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0095f0 con 0x7f06fc1014c0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070119b700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f070099a700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00b620 tx=0x7f06ec005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec01d070 con 0x7f06fc1014c0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc10bc80 con 0x7f06fc1014c0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ec021410 con 0x7f06fc1014c0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.330+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec017650 con 0x7f06fc1014c0 2023-11-17T00:06:18.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.331+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc10baf0 con 0x7f06fc1014c0 2023-11-17T00:06:18.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.332+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc1a06b0 con 0x7f06fc1014c0 2023-11-17T00:06:18.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.334+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f06ec021580 con 0x7f06fc1014c0 2023-11-17T00:06:18.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.335+0000 7f06f27fc700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 0x7f06e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.335+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06ec0ce680 con 0x7f06fc1014c0 2023-11-17T00:06:18.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.335+0000 7f070119b700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 0x7f06e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.336+0000 7f070119b700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 0x7f06e80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06f8005950 tx=0x7f06f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:18.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.337+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f06ec097c10 con 0x7f06fc1014c0 2023-11-17T00:06:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-17T00:06:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: osdmap e87: 12 total, 12 up, 12 in 2023-11-17T00:06:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-17T00:06:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: osdmap e88: 12 total, 12 up, 12 in 2023-11-17T00:06:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-17T00:06:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:18 smithi087 ceph-mon[131182]: pgmap v296: 129 pgs: 9 creating+peering, 104 active+clean, 16 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2023-11-17T00:06:18.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-17T00:06:18.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: osdmap e87: 12 total, 12 up, 12 in 2023-11-17T00:06:18.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-17T00:06:18.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: osdmap e88: 12 total, 12 up, 12 in 2023-11-17T00:06:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-17T00:06:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:18 smithi186 ceph-mon[131487]: pgmap v296: 129 pgs: 9 creating+peering, 104 active+clean, 16 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2023-11-17T00:06:18.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/240289637' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: osdmap e87: 12 total, 12 up, 12 in 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: osdmap e88: 12 total, 12 up, 12 in 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3221843889' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:18.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:18 smithi006 ceph-mon[97840]: pgmap v296: 129 pgs: 9 creating+peering, 104 active+clean, 16 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2023-11-17T00:06:18.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.547+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06fc1a0c60 con 0x7f06fc1014c0 2023-11-17T00:06:18.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.550+0000 7f06f27fc700 1 -- 172.21.15.6:0/2642989192 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f06ec09b230 con 0x7f06fc1014c0 2023-11-17T00:06:18.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:18.552 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:18.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 msgr2=0x7f06e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 0x7f06e80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06f8005950 tx=0x7f06f8009260 comp rx=0 tx=0).stop 2023-11-17T00:06:18.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 msgr2=0x7f06fc1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00b620 tx=0x7f06ec005740 comp rx=0 tx=0).stop 2023-11-17T00:06:18.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 shutdown_connections 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06e80a1150 0x7f06e80a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06fc1002d0 0x7f06fc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06fc101e50 0x7f06fc19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 --2- 172.21.15.6:0/2642989192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06fc1014c0 0x7f06fc1987a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.552+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 >> 172.21.15.6:0/2642989192 conn(0x7f06fc0fb8b0 msgr2=0x7f06fc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.553+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 shutdown_connections 2023-11-17T00:06:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.553+0000 7f07033ff700 1 -- 172.21.15.6:0/2642989192 wait complete. 2023-11-17T00:06:18.575 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 -- 172.21.15.6:0/2715006779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32281002d0 msgr2=0x7f32281006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2715006779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32281002d0 0x7f32281006e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3224009990 tx=0x7f3224009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 -- 172.21.15.6:0/2715006779 shutdown_connections 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2715006779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3228101e50 0x7f3228108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2715006779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32281002d0 0x7f32281006e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2715006779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 -- 172.21.15.6:0/2715006779 >> 172.21.15.6:0/2715006779 conn(0x7f32280fb8b0 msgr2=0x7f32280fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:18.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 -- 172.21.15.6:0/2715006779 shutdown_connections 2023-11-17T00:06:18.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.780+0000 7f322f5ff700 1 -- 172.21.15.6:0/2715006779 wait complete. 2023-11-17T00:06:18.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.781+0000 7f322f5ff700 1 Processor -- start 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.781+0000 7f322f5ff700 1 -- start start 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32281002d0 0x7f3228196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3228101e50 0x7f322819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819dba0 con 0x7f32281014c0 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819dce0 con 0x7f32281002d0 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322f5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819de70 con 0x7f3228101e50 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46946/0 (socket says 172.21.15.6:46946) 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 -- 172.21.15.6:0/2161524822 learned_addr learned my addr 172.21.15.6:0/2161524822 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322d39b700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32281002d0 0x7f3228196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322db9c700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3228101e50 0x7f322819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 -- 172.21.15.6:0/2161524822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3228101e50 msgr2=0x7f322819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3228101e50 0x7f322819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 -- 172.21.15.6:0/2161524822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32281002d0 msgr2=0x7f3228196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32281002d0 0x7f3228196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.782+0000 7f322cb9a700 1 -- 172.21.15.6:0/2161524822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218009520 con 0x7f32281014c0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f322cb9a700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f321800eb30 tx=0x7f321800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218009880 con 0x7f32281014c0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240095f0 con 0x7f32281014c0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32180044e0 con 0x7f32281014c0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003d80 con 0x7f32281014c0 2023-11-17T00:06:18.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.783+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322810bc20 con 0x7f32281014c0 2023-11-17T00:06:18.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.784+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322810bdb0 con 0x7f32281014c0 2023-11-17T00:06:18.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.786+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3218003f20 con 0x7f32281014c0 2023-11-17T00:06:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.788+0000 7f321e7fc700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:18.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.788+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3218014070 con 0x7f32281014c0 2023-11-17T00:06:18.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.789+0000 7f322d39b700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:18.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.789+0000 7f322d39b700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 0x7f32140a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f322400b2e0 tx=0x7f3224005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:18.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:18.793+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3218096590 con 0x7f32281014c0 2023-11-17T00:06:19.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.004+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3228107870 con 0x7f32281014c0 2023-11-17T00:06:19.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.005+0000 7f321e7fc700 1 -- 172.21.15.6:0/2161524822 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f321807fd50 con 0x7f32281014c0 2023-11-17T00:06:19.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:19.008 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 msgr2=0x7f32140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 0x7f32140a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f322400b2e0 tx=0x7f3224005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 msgr2=0x7f3228196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f321800eb30 tx=0x7f321800ee40 comp rx=0 tx=0).stop 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 shutdown_connections 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3228101e50 0x7f322819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32281002d0 0x7f3228196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 --2- 172.21.15.6:0/2161524822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32281014c0 0x7f3228196560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 >> 172.21.15.6:0/2161524822 conn(0x7f32280fb8b0 msgr2=0x7f32281049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:19.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 shutdown_connections 2023-11-17T00:06:19.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.008+0000 7f322f5ff700 1 -- 172.21.15.6:0/2161524822 wait complete. 2023-11-17T00:06:19.089 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1196460816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c4101e50 msgr2=0x7fc5c4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1196460816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c4101e50 0x7fc5c4108710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00b0a0 tx=0x7fc5bc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1196460816 shutdown_connections 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1196460816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c4101e50 0x7fc5c4108710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1196460816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c41002d0 0x7fc5c41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1196460816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41014c0 0x7fc5c4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1196460816 >> 172.21.15.6:0/1196460816 conn(0x7fc5c40fb8b0 msgr2=0x7fc5c40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1196460816 shutdown_connections 2023-11-17T00:06:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.292+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1196460816 wait complete. 2023-11-17T00:06:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.293+0000 7fc5c9fff700 1 Processor -- start 2023-11-17T00:06:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.293+0000 7fc5c9fff700 1 -- start start 2023-11-17T00:06:19.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41002d0 0x7fc5c4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c4101e50 0x7fc5c419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419fe90 con 0x7fc5c41014c0 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419ffd0 con 0x7fc5c4101e50 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c9fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41a0160 con 0x7fc5c41002d0 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41948/0 (socket says 172.21.15.6:41948) 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41002d0 0x7fc5c4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/1838679210 learned_addr learned my addr 172.21.15.6:0/1838679210 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c3fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c4101e50 0x7fc5c419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/1838679210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41002d0 msgr2=0x7fc5c4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41002d0 0x7fc5c4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/1838679210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c4101e50 msgr2=0x7fc5c419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c4101e50 0x7fc5c419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.294+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/1838679210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc009d00 con 0x7fc5c41014c0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9a0 tx=0x7fc5b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8009790 con 0x7fc5c41014c0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b8010430 con 0x7fc5c41014c0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c410bca0 con 0x7fc5c41014c0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8018680 con 0x7fc5c41014c0 2023-11-17T00:06:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.295+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c410bb10 con 0x7fc5c41014c0 2023-11-17T00:06:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.296+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c41a0800 con 0x7fc5c41014c0 2023-11-17T00:06:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.298+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc5b80105a0 con 0x7fc5c41014c0 2023-11-17T00:06:19.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.299+0000 7fc5c0ff9700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 0x7fc5ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.300+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc5b8014070 con 0x7fc5c41014c0 2023-11-17T00:06:19.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.300+0000 7fc5c37fe700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 0x7fc5ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.300+0000 7fc5c37fe700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 0x7fc5ac0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4101320 tx=0x7fc5b4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:19.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.303+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc5b8051730 con 0x7fc5c41014c0 2023-11-17T00:06:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/803097437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2642989192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2161524822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/803097437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2642989192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2161524822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/803097437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2642989192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2161524822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:19.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.560+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5c41a0ea0 con 0x7fc5c41014c0 2023-11-17T00:06:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.561+0000 7fc5c0ff9700 1 -- 172.21.15.6:0/1838679210 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10726 (secure 0 0 0) 0x7fc5b8096220 con 0x7fc5c41014c0 2023-11-17T00:06:19.563 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:19.563 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:17.601623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-17T00:06:19.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.563+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 msgr2=0x7fc5ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.563+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 0x7fc5ac0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4101320 tx=0x7fc5b4009260 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.563+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 msgr2=0x7fc5c4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.563+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9a0 tx=0x7fc5b800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 shutdown_connections 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc5ac0a1150 0x7fc5ac0a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5c4101e50 0x7fc5c419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5c41002d0 0x7fc5c4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 --2- 172.21.15.6:0/1838679210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c41014c0 0x7fc5c4198850 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 >> 172.21.15.6:0/1838679210 conn(0x7fc5c40fb8b0 msgr2=0x7fc5c4106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:19.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 shutdown_connections 2023-11-17T00:06:19.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.564+0000 7fc5c9fff700 1 -- 172.21.15.6:0/1838679210 wait complete. 2023-11-17T00:06:19.567 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-17T00:06:19.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:19.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 -- 172.21.15.6:0/2426222211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 msgr2=0x7fc684101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 --2- 172.21.15.6:0/2426222211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684101910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc678009990 tx=0x7fc678009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 -- 172.21.15.6:0/2426222211 shutdown_connections 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 --2- 172.21.15.6:0/2426222211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6841006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 --2- 172.21.15.6:0/2426222211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 0x7fc684108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 --2- 172.21.15.6:0/2426222211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684101910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 -- 172.21.15.6:0/2426222211 >> 172.21.15.6:0/2426222211 conn(0x7fc6840fb8b0 msgr2=0x7fc6840fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 -- 172.21.15.6:0/2426222211 shutdown_connections 2023-11-17T00:06:19.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.788+0000 7fc689fff700 1 -- 172.21.15.6:0/2426222211 wait complete. 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.789+0000 7fc689fff700 1 Processor -- start 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.789+0000 7fc689fff700 1 -- start start 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 0x7fc684073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc684073bc0 con 0x7fc6841002d0 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc684073d30 con 0x7fc6841014c0 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc689fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a0070 con 0x7fc684101e50 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41956/0 (socket says 172.21.15.6:41956) 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 -- 172.21.15.6:0/223796612 learned_addr learned my addr 172.21.15.6:0/223796612 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc682ffd700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc683fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 0x7fc684073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 -- 172.21.15.6:0/223796612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 msgr2=0x7fc684073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 0x7fc684073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 -- 172.21.15.6:0/223796612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 msgr2=0x7fc684072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.790+0000 7fc6837fe700 1 -- 172.21.15.6:0/223796612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780095f0 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc6837fe700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc67400e970 tx=0x7fc67400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674009790 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67400c7e0 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68410bc40 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674003c70 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.791+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68410bab0 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.792+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6841a0810 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.794+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc67400cc50 con 0x7fc6841002d0 2023-11-17T00:06:19.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.796+0000 7fc680ff9700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 0x7fc66c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:19.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.796+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc674014070 con 0x7fc6841002d0 2023-11-17T00:06:19.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.797+0000 7fc682ffd700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 0x7fc66c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:19.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.797+0000 7fc682ffd700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 0x7fc66c0a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b620 tx=0x7fc678005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:19.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:19.800+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc6740961a0 con 0x7fc6841002d0 2023-11-17T00:06:20.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.012+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6841071d0 con 0x7fc6841002d0 2023-11-17T00:06:20.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.014+0000 7fc680ff9700 1 -- 172.21.15.6:0/223796612 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fc674003dd0 con 0x7fc6841002d0 2023-11-17T00:06:20.016 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:20.016 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 msgr2=0x7fc66c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 0x7fc66c0a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b620 tx=0x7fc678005960 comp rx=0 tx=0).stop 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 msgr2=0x7fc6840748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc67400e970 tx=0x7fc67400ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 shutdown_connections 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc66c0a11d0 0x7fc66c0a3680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6841002d0 0x7fc6840748a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc684101e50 0x7fc684073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 --2- 172.21.15.6:0/223796612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6841014c0 0x7fc684072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 >> 172.21.15.6:0/223796612 conn(0x7fc6840fb8b0 msgr2=0x7fc6840fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 shutdown_connections 2023-11-17T00:06:20.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.016+0000 7fc689fff700 1 -- 172.21.15.6:0/223796612 wait complete. 2023-11-17T00:06:20.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-17T00:06:12.598183+0000', 'modified': '2023-11-17T00:06:17.601623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.87:6839/804288298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 804288298}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 804288298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-17T00:06:20.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-17T00:06:20.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 -- 172.21.15.6:0/1832082999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 msgr2=0x7f8214100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 --2- 172.21.15.6:0/1832082999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f8214100640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8210009990 tx=0x7f8210009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 -- 172.21.15.6:0/1832082999 shutdown_connections 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 --2- 172.21.15.6:0/1832082999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f8214100640 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 --2- 172.21.15.6:0/1832082999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101db0 0x7f8214108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 --2- 172.21.15.6:0/1832082999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101420 0x7f8214101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 -- 172.21.15.6:0/1832082999 >> 172.21.15.6:0/1832082999 conn(0x7f82140fb810 msgr2=0x7f82140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 -- 172.21.15.6:0/1832082999 shutdown_connections 2023-11-17T00:06:20.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.247+0000 7f821c7ff700 1 -- 172.21.15.6:0/1832082999 wait complete. 2023-11-17T00:06:20.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 Processor -- start 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 -- start start 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 0x7f8214198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 0x7f821419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419fd50 con 0x7f8214100230 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419fe90 con 0x7f8214101db0 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821c7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a0020 con 0x7f8214101420 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.248+0000 7f821a59b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f821ad9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 0x7f821419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 0x7f8214198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f821ad9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 0x7f821419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38034/0 (socket says 172.21.15.6:38034) 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f821a59b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41968/0 (socket says 172.21.15.6:41968) 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f821ad9c700 1 -- 172.21.15.6:0/3179611055 learned_addr learned my addr 172.21.15.6:0/3179611055 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 -- 172.21.15.6:0/3179611055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 msgr2=0x7f821419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 0x7f821419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 -- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 msgr2=0x7f82141981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 -- 172.21.15.6:0/3179611055 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009520 con 0x7f8214101420 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f821a59b700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 0x7f8214198710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9f0 tx=0x7f820400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.249+0000 7f820b7fe700 1 -- 172.21.15.6:0/3179611055 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820400cb70 con 0x7f8214101420 2023-11-17T00:06:20.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.250+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82100095f0 con 0x7f8214101420 2023-11-17T00:06:20.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.250+0000 7f820b7fe700 1 -- 172.21.15.6:0/3179611055 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820400ee90 con 0x7f8214101420 2023-11-17T00:06:20.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.250+0000 7f820b7fe700 1 -- 172.21.15.6:0/3179611055 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204003cf0 con 0x7f8214101420 2023-11-17T00:06:20.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.250+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821410bbe0 con 0x7f8214101420 2023-11-17T00:06:20.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.254+0000 7f820b7fe700 1 -- 172.21.15.6:0/3179611055 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f820400ccd0 con 0x7f8214101420 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.255+0000 7f82301a2b80 1 Processor -- start 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.255+0000 7f82301a2b80 1 -- start start 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbaf0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbaf1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb9f6010 con 0x5624bbaf20c0 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bba48850 con 0x5624bbaf1500 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f82301a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bba41ce0 con 0x5624bbaf0bb0 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f820b7fe700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbaf1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41970/0 (socket says 172.21.15.6:41970) 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f8219d9a700 1 -- 172.21.15.6:0/2592437812 learned_addr learned my addr 172.21.15.6:0/2592437812 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821ad9c700 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbaf0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821ad9c700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f820b7fe700 1 -- 172.21.15.6:0/3179611055 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8204014070 con 0x7f8214101420 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 -- 172.21.15.6:0/2592437812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 msgr2=0x5624bbaf0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbaf0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 -- 172.21.15.6:0/2592437812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 msgr2=0x5624bbaf2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f821a59b700 1 -- 172.21.15.6:0/2592437812 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624bb8c35e0 con 0x5624bbaf1500 2023-11-17T00:06:20.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.256+0000 7f8219d9a700 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:20.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.257+0000 7f821a59b700 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbaf1ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82100098e0 tx=0x7f8210005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945b0ece7ae856e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.257+0000 7f821ad9c700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 0x7f82000a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f820c009d30 tx=0x7f820c00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.257+0000 7f8208ff9700 1 -- 172.21.15.6:0/2592437812 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821001d050 con 0x5624bbaf1500 2023-11-17T00:06:20.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.257+0000 7f8208ff9700 1 -- 172.21.15.6:0/2592437812 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821001e040 con 0x5624bbaf1500 2023-11-17T00:06:20.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.257+0000 7f8208ff9700 1 -- 172.21.15.6:0/2592437812 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210005570 con 0x5624bbaf1500 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- 172.21.15.6:0/2592437812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 msgr2=0x5624bbaf1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbaf1ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82100098e0 tx=0x7f8210005ea0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- 172.21.15.6:0/2592437812 shutdown_connections 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbaf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbaf0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 --2- 172.21.15.6:0/2592437812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbaf1ae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- 172.21.15.6:0/2592437812 >> 172.21.15.6:0/2592437812 conn(0x5624bbae6310 msgr2=0x5624bbaeb620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- 172.21.15.6:0/2592437812 shutdown_connections 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- 172.21.15.6:0/2592437812 wait complete. 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.258+0000 7f82301a2b80 1 -- start start 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 Processor -- start 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb9e3890 con 0x5624bbaf20c0 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb9e3530 con 0x5624bbaf1500 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb8c58d0 con 0x5624bbaf0bb0 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f82301a2b80 10 client.0 _ll_register_callbacks cb 0x5624bba01730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:06:20.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f821a59b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38048/0 (socket says 172.21.15.6:38048) 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f8219d9a700 1 -- 172.21.15.6:0/3291486544 learned_addr learned my addr 172.21.15.6:0/3291486544 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.259+0000 7f821ad9c700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 -- 172.21.15.6:0/3291486544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 msgr2=0x5624bbafa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 -- 172.21.15.6:0/3291486544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 msgr2=0x5624bbb01a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 -- 172.21.15.6:0/3291486544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8214108fa0 con 0x5624bbaf1500 2023-11-17T00:06:20.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f8219d9a700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8204003730 tx=0x7f82040999b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148a13c198061127 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f81fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.260+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820409a430 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8204099fd0 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624bbae3ad0 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820409e000 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 10 client.24686 fetch_fsmap learned FSMap version 16 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5624bba08140 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624bbae3ad0 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8204099610 con 0x5624bbaf1500 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 10 client.24686 fetch_fsmap finished waiting for FSMap version 16 2023-11-17T00:06:20.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 10 client.24686 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 20 client.24686 populate_metadata read hostname 'smithi006' 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb057a0 0x5624bbb04ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 4 client.24686 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.261+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624bba04050 con 0x5624bbb057a0 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.262+0000 7f821a59b700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb057a0 0x5624bbb04ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.262+0000 7f821a59b700 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb057a0 0x5624bbb04ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32a4520cb9a5173 server_cookie=80c4edd8fea51b5b in_seq=0 out_seq=0 2023-11-17T00:06:20.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.262+0000 7f81fdffb700 10 client.24686 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f81fdffb700 1 -- 172.21.15.6:0/3291486544 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624bba04050 con 0x5624bbb057a0 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f81fdffb700 10 client.24686 handle_command_reply: tid=0 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 client.24686 shutdown 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 msgr2=0x5624bbafaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8204003730 tx=0x7f82040999b0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 shutdown_connections 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb057a0 0x5624bbb04ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 --2- 172.21.15.6:0/3291486544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf1500 0x5624bbafaa50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f81fdffb700 0 client.24686 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 >> 172.21.15.6:0/3291486544 conn(0x5624bbae6310 msgr2=0x5624bbaead90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 shutdown_connections 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.263+0000 7f82301a2b80 1 -- 172.21.15.6:0/3291486544 wait complete. 2023-11-17T00:06:20.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.264+0000 7f82301a2b80 20 client.24686 trim_cache size 0 max 16384 2023-11-17T00:06:20.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.282+0000 7f82301a2b80 1 Processor -- start 2023-11-17T00:06:20.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.282+0000 7f82301a2b80 1 -- start start 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.282+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f82301a2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb94ded0 con 0x5624bbaf1500 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f82301a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb9dde10 con 0x5624bbaf0bb0 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f82301a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bb8c7a30 con 0x5624bbaf20c0 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38050/0 (socket says 172.21.15.6:38050) 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42000/0 (socket says 172.21.15.6:42000) 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 -- 172.21.15.6:0/1052377068 learned_addr learned my addr 172.21.15.6:0/1052377068 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821ad9c700 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 -- 172.21.15.6:0/1052377068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf20c0 msgr2=0x5624bbb01a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 -- 172.21.15.6:0/1052377068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 msgr2=0x5624bbb050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 -- 172.21.15.6:0/1052377068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624bba07e90 con 0x5624bbaf0bb0 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f8219d9a700 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.283+0000 7f821a59b700 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82100098e0 tx=0x7f8210005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=787c157ed2dff78b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82097fa700 1 -- 172.21.15.6:0/1052377068 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821001d040 con 0x5624bbaf0bb0 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82097fa700 1 -- 172.21.15.6:0/1052377068 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821000bd40 con 0x5624bbaf0bb0 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82097fa700 1 -- 172.21.15.6:0/1052377068 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210004a40 con 0x5624bbaf0bb0 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 -- 172.21.15.6:0/1052377068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 msgr2=0x5624bbb04ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82100098e0 tx=0x7f8210005f10 comp rx=0 tx=0).stop 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 -- 172.21.15.6:0/1052377068 shutdown_connections 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf20c0 0x5624bbb01a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf0bb0 0x5624bbb04ba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 --2- 172.21.15.6:0/1052377068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbb050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 -- 172.21.15.6:0/1052377068 >> 172.21.15.6:0/1052377068 conn(0x5624bbae6310 msgr2=0x5624bbb0e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 -- 172.21.15.6:0/1052377068 shutdown_connections 2023-11-17T00:06:20.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.284+0000 7f82301a2b80 1 -- 172.21.15.6:0/1052377068 wait complete. 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.285+0000 7f82301a2b80 1 -- start start 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.285+0000 7f82301a2b80 1 Processor -- start 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 0x5624bbb59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bbb01c60 con 0x5624bbaf1500 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bbb01dd0 con 0x5624bbaf20c0 2023-11-17T00:06:20.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624bbb59ba0 con 0x5624bbaf0bb0 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f82301a2b80 10 client.0 _ll_register_callbacks cb 0x7f8204099b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42006/0 (socket says 172.21.15.6:42006) 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 -- 172.21.15.6:0/4253630750 learned_addr learned my addr 172.21.15.6:0/4253630750 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f821a59b700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f821ad9c700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 0x5624bbb59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 -- 172.21.15.6:0/4253630750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 msgr2=0x5624bbafc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 -- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 msgr2=0x5624bbb59660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:06:20.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 0x5624bbb59660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f8219d9a700 1 -- 172.21.15.6:0/4253630750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624bba08140 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.286+0000 7f821ad9c700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 0x5624bbb59660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f8219d9a700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f820400c5b0 tx=0x7f82040b84d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4371b9beb0ed7858 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204098f30 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8204099fd0 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624bba10610 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82040a2220 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f82301a2b80 10 client.14640 fetch_fsmap learned FSMap version 16 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5624bb8c35e0 con 0x5624bbaf1500 2023-11-17T00:06:20.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624bba10610 con 0x5624bbaf1500 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.287+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f820409a230 con 0x5624bbaf1500 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 10 client.14640 fetch_fsmap finished waiting for FSMap version 16 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 10 client.14640 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 20 client.14640 populate_metadata read hostname 'smithi006' 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb5e470 0x5624bbb5adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 4 client.14640 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5624bb890f20 con 0x5624bbb5e470 2023-11-17T00:06:20.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.288+0000 7f821a59b700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb5e470 0x5624bbb5adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:20.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f821a59b700 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb5e470 0x5624bbb5adc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90abac9caaf65aad server_cookie=10a9bfbd2e2f7660 in_seq=0 out_seq=0 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f81fdffb700 10 client.14640 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f81fdffb700 1 -- 172.21.15.6:0/4253630750 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5624bb890f20 con 0x5624bbb5e470 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f81fdffb700 10 client.14640 handle_command_reply: tid=0 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 client.14640 shutdown 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 msgr2=0x5624bbafcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f820400c5b0 tx=0x7f82040b84d0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 shutdown_connections 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5624bbaf20c0 0x5624bbb59660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5624bbaf0bb0 0x5624bbafc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5624bbb5e470 0x5624bbb5adc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 --2- 172.21.15.6:0/4253630750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5624bbaf1500 0x5624bbafcc30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 >> 172.21.15.6:0/4253630750 conn(0x5624bbae6310 msgr2=0x5624bbaf4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f81fdffb700 0 client.14640 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 shutdown_connections 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 1 -- 172.21.15.6:0/4253630750 wait complete. 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.289+0000 7f82301a2b80 20 client.14640 trim_cache size 0 max 16384 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:06:20.299 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dbc2f8cc-84db-11ee-8dbe-212e2dc638e7", 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24578, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 7.6885347560000001, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 16, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 88, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 87, 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 19.848986427 2023-11-17T00:06:20.300 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:06:20.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 msgr2=0x7f82000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 0x7f82000a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f820c009d30 tx=0x7f820c00b040 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 msgr2=0x7f8214198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 0x7f8214198710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9f0 tx=0x7f820400ed00 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 shutdown_connections 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8214100230 0x7f82141981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8214101db0 0x7f821419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 --2- 172.21.15.6:0/3179611055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8214101420 0x7f8214198710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 >> 172.21.15.6:0/3179611055 conn(0x7f82140fb810 msgr2=0x7f82140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 shutdown_connections 2023-11-17T00:06:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:20.290+0000 7f821c7ff700 1 -- 172.21.15.6:0/3179611055 wait complete. 2023-11-17T00:06:20.307 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:20 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1838679210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:20.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:20 smithi006 ceph-mon[97840]: pgmap v297: 129 pgs: 9 creating+peering, 115 active+clean, 5 unknown; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 5 op/s 2023-11-17T00:06:20.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:20 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/223796612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:20.310 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-17T00:06:20.310 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:20.310 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph fs ls 2023-11-17T00:06:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:20 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1838679210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:20 smithi087 ceph-mon[131182]: pgmap v297: 129 pgs: 9 creating+peering, 115 active+clean, 5 unknown; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 5 op/s 2023-11-17T00:06:20.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:20 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/223796612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:20.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:20 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1838679210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:20.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:20 smithi186 ceph-mon[131487]: pgmap v297: 129 pgs: 9 creating+peering, 115 active+clean, 5 unknown; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 5 op/s 2023-11-17T00:06:20.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:20 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/223796612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:21.172 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:22.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 -- 172.21.15.6:0/1370092083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 msgr2=0x7f8434100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:22.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 --2- 172.21.15.6:0/1370092083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434100910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84240099c0 tx=0x7f8424009cd0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 -- 172.21.15.6:0/1370092083 shutdown_connections 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 --2- 172.21.15.6:0/1370092083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434100910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 --2- 172.21.15.6:0/1370092083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84341055e0 0x7f843410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 --2- 172.21.15.6:0/1370092083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434100ee0 0x7f8434104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 -- 172.21.15.6:0/1370092083 >> 172.21.15.6:0/1370092083 conn(0x7f84340fbdc0 msgr2=0x7f84340fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 -- 172.21.15.6:0/1370092083 shutdown_connections 2023-11-17T00:06:22.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.703+0000 7f843a282700 1 -- 172.21.15.6:0/1370092083 wait complete. 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.704+0000 7f843a282700 1 Processor -- start 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.704+0000 7f843a282700 1 -- start start 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8434100ee0 0x7f8434073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84341055e0 0x7f84340738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84341a4810 con 0x7f8434100540 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84341a4980 con 0x7f84341055e0 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843a282700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84341a4b40 con 0x7f8434100ee0 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f8433fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:22.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f8433fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42016/0 (socket says 172.21.15.6:42016) 2023-11-17T00:06:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f8433fff700 1 -- 172.21.15.6:0/710438700 learned_addr learned my addr 172.21.15.6:0/710438700 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f84337fe700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8434100ee0 0x7f8434073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.705+0000 7f843881f700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84341055e0 0x7f84340738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:22.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 -- 172.21.15.6:0/710438700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8434100ee0 msgr2=0x7f8434073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8434100ee0 0x7f8434073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 -- 172.21.15.6:0/710438700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84341055e0 msgr2=0x7f84340738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84341055e0 0x7f84340738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 -- 172.21.15.6:0/710438700 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240095f0 con 0x7f8434100540 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f8433fff700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8424000c00 tx=0x7f842400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842401d070 con 0x7f8434100540 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8434110320 con 0x7f8434100540 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8424005050 con 0x7f8434100540 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424004b70 con 0x7f8434100540 2023-11-17T00:06:22.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.706+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434110190 con 0x7f8434100540 2023-11-17T00:06:22.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.707+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84341104b0 con 0x7f8434100540 2023-11-17T00:06:22.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.709+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8424004cd0 con 0x7f8434100540 2023-11-17T00:06:22.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.711+0000 7f84317fa700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 0x7f841c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:22.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.711+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84240cd8d0 con 0x7f8434100540 2023-11-17T00:06:22.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.711+0000 7f84337fe700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 0x7f841c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:22.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.711+0000 7f84337fe700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 0x7f841c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84340745e0 tx=0x7f8428009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:22.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.714+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8424096e60 con 0x7f8434100540 2023-11-17T00:06:22.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.969+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8434066ea0 con 0x7f8434100540 2023-11-17T00:06:22.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.970+0000 7f84317fa700 1 -- 172.21.15.6:0/710438700 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8424026070 con 0x7f8434100540 2023-11-17T00:06:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 msgr2=0x7f841c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 0x7f841c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84340745e0 tx=0x7f8428009260 comp rx=0 tx=0).stop 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 msgr2=0x7f8434072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8424000c00 tx=0x7f842400f690 comp rx=0 tx=0).stop 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 shutdown_connections 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f841c0a1150 0x7f841c0a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8434100540 0x7f8434072df0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84341055e0 0x7f84340738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 --2- 172.21.15.6:0/710438700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8434100ee0 0x7f8434073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 >> 172.21.15.6:0/710438700 conn(0x7f84340fbdc0 msgr2=0x7f84340fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 shutdown_connections 2023-11-17T00:06:22.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:22.972+0000 7f843a282700 1 -- 172.21.15.6:0/710438700 wait complete. 2023-11-17T00:06:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:22 smithi087 ceph-mon[131182]: pgmap v298: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2023-11-17T00:06:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:22 smithi186 ceph-mon[131487]: pgmap v298: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2023-11-17T00:06:23.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:22 smithi006 ceph-mon[97840]: pgmap v298: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2023-11-17T00:06:23.453 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-17T00:06:23.454 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-17T00:06:23.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:23.455 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-17T00:06:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2023-11-17T00:06:23.500 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 41642sec preferred_lft 41642sec 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:06:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2023-11-17T00:06:23.503 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-17T00:06:23.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:23.503 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:06:23.503 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:06:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2023-11-17T00:06:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2023-11-17T00:06:23.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2023-11-17T00:06:23.504 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-17T00:06:23.504 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:06:23.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:23.977 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-17T00:06:24.047 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:23 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/710438700' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:06:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-17T00:06:24.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:24.056 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-17T00:06:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:23 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/710438700' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:06:24.111 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-17T00:06:24.111 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-17T00:06:24.111 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-17T00:06:24.111 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-17T00:06:24.111 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-17T00:06:24.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-17T00:06:24.113 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:06:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:23 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/710438700' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:06:24.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:24.819 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-17T00:06:24.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:24.864 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-17T00:06:24.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:24.948 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:06:24.948 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:06:24.948 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-17T00:06:24.948 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-17T00:06:24.948 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:06:25.092 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:24 smithi006 ceph-mon[97840]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 266 B/s wr, 3 op/s 2023-11-17T00:06:25.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:24 smithi087 ceph-mon[131182]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 266 B/s wr, 3 op/s 2023-11-17T00:06:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:24 smithi186 ceph-mon[131487]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 266 B/s wr, 3 op/s 2023-11-17T00:06:25.140 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-17T00:06:25.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:25.140 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:06:25.140 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-17T00:06:25.141 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:06:25.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:25.433 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:06:25.433 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:06:25.433 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-17T00:06:25.433 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2023-11-17T00:06:25.433 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:06:25.611 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-17T00:06:25.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-17T00:06:25.611 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:25.656 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-17T00:06:25.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-17T00:06:25.657 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:25.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-17T00:06:25.854 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-17T00:06:25.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-17T00:06:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-17T00:06:25.926 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-17T00:06:25.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:25.927 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-17T00:06:25.971 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-17T00:06:25.972 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-17T00:06:26.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-17T00:06:26.089 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-17T00:06:26.089 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-17T00:06:26.162 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-17T00:06:26.165 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-17T00:06:26.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:26.166 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-17T00:06:26.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:2023-11-17T00:06:26.214+0000 7f82227805c0 -1 init, newargv = 0x555e1ffba500 newargc=15 2023-11-17T00:06:26.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[134438]: starting ceph client 2023-11-17T00:06:26.225 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[134438]: starting fuse 2023-11-17T00:06:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:26 smithi087 ceph-mon[131182]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-17T00:06:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:26 smithi186 ceph-mon[131487]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-17T00:06:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:26 smithi006 ceph-mon[97840]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-17T00:06:27.310 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-17T00:06:27.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-17T00:06:27.391 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-17T00:06:27.462 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-17T00:06:27.465 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-17T00:06:27.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:27.465 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-17T00:06:27.509 INFO:teuthology.orchestra.run.smithi006.stdout:121 2023-11-17T00:06:27.509 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-17T00:06:27.510 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.admin")) 2023-11-17T00:06:27.511 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-17T00:06:27.696 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.admin.134438.asok 2023-11-17T00:06:27.874 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.134438.asok 2023-11-17T00:06:27.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:27.874 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.134438.asok status 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "1c9565146c8647d4e00eb6c7ae626dc192680e7b", 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "admin", 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-17T00:06:28.110 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "134438", 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24682, 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 24682 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:06:28.111 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1849445040 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1849445040 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.24682 172.21.15.6:0/1849445040", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "172.21.15.6:0/1849445040", 2023-11-17T00:06:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 16, 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 88, 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-17T00:06:28.113 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:06:28.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:28.123 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-17T00:06:28.123 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:28.168 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-17T00:06:28.168 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:28.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:28.169 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:28.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:28.280 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-17T00:06:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:# file: . 2023-11-17T00:06:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-17T00:06:28.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:28.407 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:06:28.407 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-17T00:06:28.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:28.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 -- 172.21.15.6:0/3394870201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba901002d0 msgr2=0x7fba901006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:28.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 --2- 172.21.15.6:0/3394870201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba901002d0 0x7fba901006e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba80009990 tx=0x7fba80009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 -- 172.21.15.6:0/3394870201 shutdown_connections 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 --2- 172.21.15.6:0/3394870201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba901002d0 0x7fba901006e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 --2- 172.21.15.6:0/3394870201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba90101e50 0x7fba90108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 --2- 172.21.15.6:0/3394870201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 0x7fba90101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.701+0000 7fba951ff700 1 -- 172.21.15.6:0/3394870201 >> 172.21.15.6:0/3394870201 conn(0x7fba900fb8b0 msgr2=0x7fba900fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.702+0000 7fba951ff700 1 -- 172.21.15.6:0/3394870201 shutdown_connections 2023-11-17T00:06:28.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.702+0000 7fba951ff700 1 -- 172.21.15.6:0/3394870201 wait complete. 2023-11-17T00:06:28.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 Processor -- start 2023-11-17T00:06:28.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 -- start start 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 0x7fba901987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9019fe10 con 0x7fba90101e50 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9019ff50 con 0x7fba901014c0 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.703+0000 7fba951ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901a00e0 con 0x7fba901002d0 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42142/0 (socket says 172.21.15.6:42142) 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42022/0 (socket says 172.21.15.6:42022) 2023-11-17T00:06:28.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8ed9d700 1 -- 172.21.15.6:0/2130474801 learned_addr learned my addr 172.21.15.6:0/2130474801 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 0x7fba901987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 -- 172.21.15.6:0/2130474801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 msgr2=0x7fba90198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 -- 172.21.15.6:0/2130474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 msgr2=0x7fba901987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 0x7fba901987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 -- 172.21.15.6:0/2130474801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84009520 con 0x7fba90101e50 2023-11-17T00:06:28.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8ed9d700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:28.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.704+0000 7fba8f59e700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba8800efc0 tx=0x7fba8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:28.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.705+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88018070 con 0x7fba90101e50 2023-11-17T00:06:28.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.705+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba8800f040 con 0x7fba90101e50 2023-11-17T00:06:28.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.705+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba800095f0 con 0x7fba90101e50 2023-11-17T00:06:28.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.705+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8800cd40 con 0x7fba90101e50 2023-11-17T00:06:28.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.705+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9010bca0 con 0x7fba90101e50 2023-11-17T00:06:28.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.706+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba90192a00 con 0x7fba90101e50 2023-11-17T00:06:28.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.708+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fba88003a00 con 0x7fba90101e50 2023-11-17T00:06:28.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.709+0000 7fba77fff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 0x7fba780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:28.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.709+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba880cc690 con 0x7fba90101e50 2023-11-17T00:06:28.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.709+0000 7fba8ed9d700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 0x7fba780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:28.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.710+0000 7fba8ed9d700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 0x7fba780a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b2e0 tx=0x7fba80005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:28.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.713+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba88095c20 con 0x7fba90101e50 2023-11-17T00:06:28.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.928+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba9002cfe0 con 0x7fba90101e50 2023-11-17T00:06:28.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.930+0000 7fba77fff700 1 -- 172.21.15.6:0/2130474801 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fba88099240 con 0x7fba90101e50 2023-11-17T00:06:28.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:28.932 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:28.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 msgr2=0x7fba780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:28.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 0x7fba780a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b2e0 tx=0x7fba80005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 msgr2=0x7fba9019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:28.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba8800efc0 tx=0x7fba8800c3a0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 shutdown_connections 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fba780a11d0 0x7fba780a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fba90101e50 0x7fba9019f840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba901002d0 0x7fba90198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 --2- 172.21.15.6:0/2130474801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba901014c0 0x7fba901987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.932+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 >> 172.21.15.6:0/2130474801 conn(0x7fba900fb8b0 msgr2=0x7fba90106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.933+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 shutdown_connections 2023-11-17T00:06:28.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:28.933+0000 7fba951ff700 1 -- 172.21.15.6:0/2130474801 wait complete. 2023-11-17T00:06:28.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:06:28.953 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:06:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:28 smithi087 ceph-mon[131182]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:28 smithi186 ceph-mon[131487]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:29.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.159+0000 7ffb09dff700 1 -- 172.21.15.6:0/1594378663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101480 msgr2=0x7ffb041018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.159+0000 7ffb09dff700 1 --2- 172.21.15.6:0/1594378663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101480 0x7ffb041018d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009990 tx=0x7ffaf8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 -- 172.21.15.6:0/1594378663 shutdown_connections 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 --2- 172.21.15.6:0/1594378663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101e10 0x7ffb041086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 --2- 172.21.15.6:0/1594378663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 --2- 172.21.15.6:0/1594378663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101480 0x7ffb041018d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 -- 172.21.15.6:0/1594378663 >> 172.21.15.6:0/1594378663 conn(0x7ffb040fb870 msgr2=0x7ffb040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 -- 172.21.15.6:0/1594378663 shutdown_connections 2023-11-17T00:06:29.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.160+0000 7ffb09dff700 1 -- 172.21.15.6:0/1594378663 wait complete. 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 Processor -- start 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 -- start start 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 0x7ffb04198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101e10 0x7ffb0419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0419fd60 con 0x7ffb04101e10 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0419fea0 con 0x7ffb04101480 2023-11-17T00:06:29.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb09dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041a0030 con 0x7ffb04100290 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.161+0000 7ffb037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59098/0 (socket says 172.21.15.6:59098) 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb037fe700 1 -- 172.21.15.6:0/2196707141 learned_addr learned my addr 172.21.15.6:0/2196707141 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 0x7ffb04198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb03fff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101e10 0x7ffb0419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 -- 172.21.15.6:0/2196707141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 msgr2=0x7ffb041981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 -- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101e10 msgr2=0x7ffb0419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101e10 0x7ffb0419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 -- 172.21.15.6:0/2196707141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80095f0 con 0x7ffb04101480 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb02ffd700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 0x7ffb04198720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800b620 tx=0x7ffaf8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:29.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf801d070 con 0x7ffb04101480 2023-11-17T00:06:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.162+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0410bc50 con 0x7ffb04101480 2023-11-17T00:06:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.163+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf8021410 con 0x7ffb04101480 2023-11-17T00:06:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.163+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8017650 con 0x7ffb04101480 2023-11-17T00:06:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.163+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0410bac0 con 0x7ffb04101480 2023-11-17T00:06:29.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.163+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb04067a50 con 0x7ffb04101480 2023-11-17T00:06:29.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.166+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffaf80217f0 con 0x7ffb04101480 2023-11-17T00:06:29.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.167+0000 7ffb00ff9700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 0x7ffaec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.167+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffaf80ce490 con 0x7ffb04101480 2023-11-17T00:06:29.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.168+0000 7ffb037fe700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 0x7ffaec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.168+0000 7ffb037fe700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 0x7ffaec0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4005950 tx=0x7ffaf4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:29.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.169+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffaf8097a20 con 0x7ffb04101480 2023-11-17T00:06:29.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:28 smithi006 ceph-mon[97840]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:06:29.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.380+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb041a0ac0 con 0x7ffb04101480 2023-11-17T00:06:29.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.381+0000 7ffb00ff9700 1 -- 172.21.15.6:0/2196707141 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffaf8021e60 con 0x7ffb04101480 2023-11-17T00:06:29.382 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 msgr2=0x7ffaec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 0x7ffaec0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4005950 tx=0x7ffaf4009260 comp rx=0 tx=0).stop 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 msgr2=0x7ffb04198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 0x7ffb04198720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800b620 tx=0x7ffaf8004730 comp rx=0 tx=0).stop 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 shutdown_connections 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffaec0a1150 0x7ffaec0a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb04101e10 0x7ffb0419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb04100290 0x7ffb041981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 --2- 172.21.15.6:0/2196707141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb04101480 0x7ffb04198720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 >> 172.21.15.6:0/2196707141 conn(0x7ffb040fb870 msgr2=0x7ffb040fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.383+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 shutdown_connections 2023-11-17T00:06:29.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.384+0000 7ffb09dff700 1 -- 172.21.15.6:0/2196707141 wait complete. 2023-11-17T00:06:29.403 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 -- 172.21.15.6:0/615192880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 msgr2=0x7fc2a0108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 --2- 172.21.15.6:0/615192880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a0108530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc298009f90 tx=0x7fc29800b330 comp rx=0 tx=0).stop 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 -- 172.21.15.6:0/615192880 shutdown_connections 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 --2- 172.21.15.6:0/615192880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a0108530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 --2- 172.21.15.6:0/615192880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a0100050 0x7fc2a0100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 --2- 172.21.15.6:0/615192880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a01012e0 0x7fc2a0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 -- 172.21.15.6:0/615192880 >> 172.21.15.6:0/615192880 conn(0x7fc2a00fb650 msgr2=0x7fc2a00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:29.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 -- 172.21.15.6:0/615192880 shutdown_connections 2023-11-17T00:06:29.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.609+0000 7fc29f59e700 1 -- 172.21.15.6:0/615192880 wait complete. 2023-11-17T00:06:29.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.610+0000 7fc29f59e700 1 Processor -- start 2023-11-17T00:06:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 -- start start 2023-11-17T00:06:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a0100050 0x7fc2a0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a01012e0 0x7fc2a0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a019fca0 con 0x7fc2a0101c90 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a019fde0 con 0x7fc2a01012e0 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29f59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a019ff70 con 0x7fc2a0100050 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46338/0 (socket says 172.21.15.6:46338) 2023-11-17T00:06:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29ed9d700 1 -- 172.21.15.6:0/3920368315 learned_addr learned my addr 172.21.15.6:0/3920368315 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29e59c700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a0100050 0x7fc2a0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.611+0000 7fc29dd9b700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a01012e0 0x7fc2a0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 -- 172.21.15.6:0/3920368315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a0100050 msgr2=0x7fc2a0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a0100050 0x7fc2a0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 -- 172.21.15.6:0/3920368315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a01012e0 msgr2=0x7fc2a0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a01012e0 0x7fc2a0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 -- 172.21.15.6:0/3920368315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294009520 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29ed9d700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc298009f60 tx=0x7fc298004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29800e040 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29800bbf0 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc298009c40 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980052a0 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.612+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a010ba10 con 0x7fc2a0101c90 2023-11-17T00:06:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.613+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a010bba0 con 0x7fc2a0101c90 2023-11-17T00:06:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.615+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc29801d470 con 0x7fc2a0101c90 2023-11-17T00:06:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.616+0000 7fc28f7fe700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 0x7fc2880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:29.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.617+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc2980ce120 con 0x7fc2a0101c90 2023-11-17T00:06:29.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.617+0000 7fc29e59c700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 0x7fc2880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:29.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.617+0000 7fc29e59c700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 0x7fc2880a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc290005fd0 tx=0x7fc290009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:29.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.620+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc2980976b0 con 0x7fc2a0101c90 2023-11-17T00:06:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.830+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2a01a0980 con 0x7fc2a0101c90 2023-11-17T00:06:29.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.830+0000 7fc28f7fe700 1 -- 172.21.15.6:0/3920368315 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc29809acd0 con 0x7fc2a0101c90 2023-11-17T00:06:29.832 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:06:29.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.832+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 msgr2=0x7fc2880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.832+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 0x7fc2880a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc290005fd0 tx=0x7fc290009310 comp rx=0 tx=0).stop 2023-11-17T00:06:29.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.832+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 msgr2=0x7fc2a019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:29.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.832+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc298009f60 tx=0x7fc298004c60 comp rx=0 tx=0).stop 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 shutdown_connections 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc2880a11d0 0x7fc2880a3680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2a0101c90 0x7fc2a019f6d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2a0100050 0x7fc2a0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 --2- 172.21.15.6:0/3920368315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2a01012e0 0x7fc2a0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 >> 172.21.15.6:0/3920368315 conn(0x7fc2a00fb650 msgr2=0x7fc2a0106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 shutdown_connections 2023-11-17T00:06:29.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:29.833+0000 7fc29f59e700 1 -- 172.21.15.6:0/3920368315 wait complete. 2023-11-17T00:06:29.853 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:06:30.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 -- 172.21.15.6:0/4080737785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a601012e0 msgr2=0x7f3a60101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/4080737785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a601012e0 0x7f3a60101750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009990 tx=0x7f3a54009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 -- 172.21.15.6:0/4080737785 shutdown_connections 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/4080737785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60101c90 0x7f3a60108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/4080737785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a60100050 0x7f3a60100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/4080737785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a601012e0 0x7f3a60101750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 -- 172.21.15.6:0/4080737785 >> 172.21.15.6:0/4080737785 conn(0x7f3a600fb650 msgr2=0x7f3a600fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.058+0000 7f3a5ffff700 1 -- 172.21.15.6:0/4080737785 shutdown_connections 2023-11-17T00:06:30.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.059+0000 7f3a5ffff700 1 -- 172.21.15.6:0/4080737785 wait complete. 2023-11-17T00:06:30.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.059+0000 7f3a5ffff700 1 Processor -- start 2023-11-17T00:06:30.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 -- start start 2023-11-17T00:06:30.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a601012e0 0x7f3a601063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 0x7f3a60102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601031a0 con 0x7f3a60101c90 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60103310 con 0x7f3a601012e0 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601034d0 con 0x7f3a60100050 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 0x7f3a60102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59140/0 (socket says 172.21.15.6:59140) 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5effd700 1 -- 172.21.15.6:0/3733799915 learned_addr learned my addr 172.21.15.6:0/3733799915 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.060+0000 7f3a5e7fc700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a601012e0 0x7f3a601063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 -- 172.21.15.6:0/3733799915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a601012e0 msgr2=0x7f3a601063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a601012e0 0x7f3a601063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 -- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 msgr2=0x7f3a60102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 0x7f3a60102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 -- 172.21.15.6:0/3733799915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50009520 con 0x7f3a60100050 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5f7fe700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 0x7f3a60102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5effd700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000eb30 tx=0x7f3a5000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50009880 con 0x7f3a60100050 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a500044e0 con 0x7f3a60100050 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50003d80 con 0x7f3a60100050 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a540095f0 con 0x7f3a60100050 2023-11-17T00:06:30.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.061+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6010ba10 con 0x7f3a60100050 2023-11-17T00:06:30.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.062+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60067970 con 0x7f3a60100050 2023-11-17T00:06:30.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.064+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3a50003f20 con 0x7f3a60100050 2023-11-17T00:06:30.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.066+0000 7f3a47fff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 0x7f3a480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.066+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a50014070 con 0x7f3a60100050 2023-11-17T00:06:30.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.066+0000 7f3a5e7fc700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 0x7f3a480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.066+0000 7f3a5e7fc700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 0x7f3a480a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b2e0 tx=0x7f3a54005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.070+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a50096540 con 0x7f3a60100050 2023-11-17T00:06:30.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:29 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2130474801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:30.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:29 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2196707141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:29 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2130474801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:29 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2196707141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:30.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:29 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2130474801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:30.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:29 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2196707141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:30.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.285+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a601a4d80 con 0x7f3a60100050 2023-11-17T00:06:30.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.286+0000 7f3a47fff700 1 -- 172.21.15.6:0/3733799915 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a5001e2e0 con 0x7f3a60100050 2023-11-17T00:06:30.289 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 msgr2=0x7f3a480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 0x7f3a480a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b2e0 tx=0x7f3a54005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 msgr2=0x7f3a60105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000eb30 tx=0x7f3a5000ee40 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 shutdown_connections 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3a480a11d0 0x7f3a480a3680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a60101c90 0x7f3a60102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a60100050 0x7f3a60105e70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 --2- 172.21.15.6:0/3733799915 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a601012e0 0x7f3a601063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 >> 172.21.15.6:0/3733799915 conn(0x7f3a600fb650 msgr2=0x7f3a600fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:30.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 shutdown_connections 2023-11-17T00:06:30.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.289+0000 7f3a5ffff700 1 -- 172.21.15.6:0/3733799915 wait complete. 2023-11-17T00:06:30.309 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:06:30.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 -- 172.21.15.6:0/1833462419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 msgr2=0x7f500c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 --2- 172.21.15.6:0/1833462419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c1006a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f50000098f0 tx=0x7f5000009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 -- 172.21.15.6:0/1833462419 shutdown_connections 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 --2- 172.21.15.6:0/1833462419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c1006a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 --2- 172.21.15.6:0/1833462419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101e10 0x7f500c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 --2- 172.21.15.6:0/1833462419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101480 0x7f500c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.515+0000 7f5012dff700 1 -- 172.21.15.6:0/1833462419 >> 172.21.15.6:0/1833462419 conn(0x7f500c0fb870 msgr2=0x7f500c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.516+0000 7f5012dff700 1 -- 172.21.15.6:0/1833462419 shutdown_connections 2023-11-17T00:06:30.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.516+0000 7f5012dff700 1 -- 172.21.15.6:0/1833462419 wait complete. 2023-11-17T00:06:30.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 Processor -- start 2023-11-17T00:06:30.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 -- start start 2023-11-17T00:06:30.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101480 0x7f500c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101e10 0x7f500c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c073bc0 con 0x7f500c100290 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c073d30 con 0x7f500c101e10 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5012dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c1a00e0 con 0x7f500c101480 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5010b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f500bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101480 0x7f500c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5010b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46370/0 (socket says 172.21.15.6:46370) 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f5010b9b700 1 -- 172.21.15.6:0/1088559849 learned_addr learned my addr 172.21.15.6:0/1088559849 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.517+0000 7f501139c700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101e10 0x7f500c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 -- 172.21.15.6:0/1088559849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101480 msgr2=0x7f500c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101480 0x7f500c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 -- 172.21.15.6:0/1088559849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101e10 msgr2=0x7f500c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101e10 0x7f500c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 -- 172.21.15.6:0/1088559849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004009d00 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5010b9b700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e90 tx=0x7f500000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500001c070 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5000009520 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f500000f7f0 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.518+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000003990 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.519+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c10bc40 con 0x7f500c100290 2023-11-17T00:06:30.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.520+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c10bdd0 con 0x7f500c100290 2023-11-17T00:06:30.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.522+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5000020030 con 0x7f500c100290 2023-11-17T00:06:30.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.524+0000 7f5009ffb700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 0x7f4ffc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.524+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f50000cdac0 con 0x7f500c100290 2023-11-17T00:06:30.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.524+0000 7f500bfff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 0x7f4ffc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.525+0000 7f500bfff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 0x7f4ffc0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009c60 tx=0x7f4ff8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.528+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5000097050 con 0x7f500c100290 2023-11-17T00:06:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.739+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f500c061840 con 0x7f500c100290 2023-11-17T00:06:30.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.740+0000 7f5009ffb700 1 -- 172.21.15.6:0/1088559849 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f500009a670 con 0x7f500c100290 2023-11-17T00:06:30.742 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:06:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 msgr2=0x7f4ffc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 0x7f4ffc0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009c60 tx=0x7f4ff8009260 comp rx=0 tx=0).stop 2023-11-17T00:06:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 msgr2=0x7f500c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e90 tx=0x7f500000b730 comp rx=0 tx=0).stop 2023-11-17T00:06:30.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 shutdown_connections 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4ffc0a1150 0x7f4ffc0a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f500c100290 0x7f500c0748a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f500c101e10 0x7f500c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 --2- 172.21.15.6:0/1088559849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f500c101480 0x7f500c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 >> 172.21.15.6:0/1088559849 conn(0x7f500c0fb870 msgr2=0x7f500c104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 shutdown_connections 2023-11-17T00:06:30.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.742+0000 7f5012dff700 1 -- 172.21.15.6:0/1088559849 wait complete. 2023-11-17T00:06:30.765 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-17T00:06:30.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.970+0000 7ff36c3ff700 1 -- 172.21.15.6:0/2302299506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364100230 msgr2=0x7ff364100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.970+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/2302299506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364100230 0x7ff364100640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff360009990 tx=0x7ff360009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 -- 172.21.15.6:0/2302299506 shutdown_connections 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/2302299506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364100230 0x7ff364100640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/2302299506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101db0 0x7ff364108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/2302299506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364101420 0x7ff364101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 -- 172.21.15.6:0/2302299506 >> 172.21.15.6:0/2302299506 conn(0x7ff3640fb810 msgr2=0x7ff3640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 -- 172.21.15.6:0/2302299506 shutdown_connections 2023-11-17T00:06:30.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.971+0000 7ff36c3ff700 1 -- 172.21.15.6:0/2302299506 wait complete. 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 Processor -- start 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 -- start start 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364100230 0x7ff3640748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101420 0x7ff364072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364073bc0 con 0x7ff364101db0 2023-11-17T00:06:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364073d30 con 0x7ff364101420 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3641a0010 con 0x7ff364100230 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36a99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36a99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46386/0 (socket says 172.21.15.6:46386) 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36a99c700 1 -- 172.21.15.6:0/1618729950 learned_addr learned my addr 172.21.15.6:0/1618729950 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.972+0000 7ff36a19b700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364100230 0x7ff3640748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36999a700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101420 0x7ff364072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 -- 172.21.15.6:0/1618729950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364100230 msgr2=0x7ff3640748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364100230 0x7ff3640748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 -- 172.21.15.6:0/1618729950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101420 msgr2=0x7ff364072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101420 0x7ff364072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 -- 172.21.15.6:0/1618729950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600095f0 con 0x7ff364101db0 2023-11-17T00:06:30.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36a99c700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007ee0 tx=0x7ff35c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c017070 con 0x7ff364101db0 2023-11-17T00:06:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35c00f040 con 0x7ff364101db0 2023-11-17T00:06:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c005580 con 0x7ff364101db0 2023-11-17T00:06:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.973+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36410bbe0 con 0x7ff364101db0 2023-11-17T00:06:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.974+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36410ba50 con 0x7ff364101db0 2023-11-17T00:06:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.975+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364107130 con 0x7ff364101db0 2023-11-17T00:06:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.976+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff35c00db20 con 0x7ff364101db0 2023-11-17T00:06:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.978+0000 7ff35b7fe700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 0x7ff3500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.978+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff35c0cc200 con 0x7ff364101db0 2023-11-17T00:06:30.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.978+0000 7ff36a19b700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 0x7ff3500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:30.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.979+0000 7ff36a19b700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 0x7ff3500a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b2e0 tx=0x7ff360005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:30.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:30.982+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff35c095700 con 0x7ff364101db0 2023-11-17T00:06:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:30 smithi087 ceph-mon[131182]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:30 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3920368315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:30 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3733799915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:30 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1088559849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:30 smithi186 ceph-mon[131487]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:31.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:30 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3920368315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:30 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3733799915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:30 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1088559849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.207+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff36402cfe0 con 0x7ff364101db0 2023-11-17T00:06:31.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.208+0000 7ff35b7fe700 1 -- 172.21.15.6:0/1618729950 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+580 (secure 0 0 0) 0x7ff35c098d20 con 0x7ff364101db0 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1821215069 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4157530954 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3407461840 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3069406217 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3467557048 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:31.210 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2971625275 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:31.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 msgr2=0x7ff3500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 0x7ff3500a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b2e0 tx=0x7ff360005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 msgr2=0x7ff364073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007ee0 tx=0x7ff35c00d3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 shutdown_connections 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff3500a11d0 0x7ff3500a3680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff364101db0 0x7ff364073420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff364100230 0x7ff3640748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.210+0000 7ff36c3ff700 1 --2- 172.21.15.6:0/1618729950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff364101420 0x7ff364072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.211+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 >> 172.21.15.6:0/1618729950 conn(0x7ff3640fb810 msgr2=0x7ff3640fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.211+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 shutdown_connections 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.211+0000 7ff36c3ff700 1 -- 172.21.15.6:0/1618729950 wait complete. 2023-11-17T00:06:31.214 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2023-11-17T00:06:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:30 smithi006 ceph-mon[97840]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:30 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3920368315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:30 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3733799915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:30 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1088559849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:31.231 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:06:31.231 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-17T00:06:31.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.518+0000 7f1f3abff700 1 -- 172.21.15.6:0/503269937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 msgr2=0x7f1f34072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.518+0000 7f1f3abff700 1 --2- 172.21.15.6:0/503269937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34072f80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b0a0 tx=0x7f1f2800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 -- 172.21.15.6:0/503269937 shutdown_connections 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 --2- 172.21.15.6:0/503269937 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34072f80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 --2- 172.21.15.6:0/503269937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 0x7f1f3410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 --2- 172.21.15.6:0/503269937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f340739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 -- 172.21.15.6:0/503269937 >> 172.21.15.6:0/503269937 conn(0x7f1f340fb8b0 msgr2=0x7f1f340fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 -- 172.21.15.6:0/503269937 shutdown_connections 2023-11-17T00:06:31.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.519+0000 7f1f3abff700 1 -- 172.21.15.6:0/503269937 wait complete. 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.520+0000 7f1f3abff700 1 Processor -- start 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.520+0000 7f1f3abff700 1 -- start start 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f34198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 0x7f1f3419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3419ffa0 con 0x7f1f34074b30 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a00e0 con 0x7f1f34108190 2023-11-17T00:06:31.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3abff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341a0270 con 0x7f1f34073550 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46390/0 (socket says 172.21.15.6:46390) 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3899b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f34198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f33fff700 1 -- 172.21.15.6:0/2308737959 learned_addr learned my addr 172.21.15.6:0/2308737959 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.521+0000 7f1f3919c700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 0x7f1f3419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 -- 172.21.15.6:0/2308737959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 msgr2=0x7f1f3419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 0x7f1f3419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 -- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 msgr2=0x7f1f34198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 -- 172.21.15.6:0/2308737959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28009d00 con 0x7f1f34073550 2023-11-17T00:06:31.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f33fff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3899b700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f34198420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b070 tx=0x7f1f2800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800e040 con 0x7f1f34073550 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3410de90 con 0x7f1f34073550 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f28003c00 con 0x7f1f34073550 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2801b6e0 con 0x7f1f34073550 2023-11-17T00:06:31.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.522+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3410dd00 con 0x7f1f34073550 2023-11-17T00:06:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.523+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f34106840 con 0x7f1f34073550 2023-11-17T00:06:31.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.525+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1f2801d470 con 0x7f1f34073550 2023-11-17T00:06:31.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.527+0000 7f1f31ffb700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.527+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f280ce4f0 con 0x7f1f34073550 2023-11-17T00:06:31.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.527+0000 7f1f33fff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.528+0000 7f1f33fff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 0x7f1f240a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f20005950 tx=0x7f1f20009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:31.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.529+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1f28097a80 con 0x7f1f34073550 2023-11-17T00:06:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.738+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f34067af0 con 0x7f1f34073550 2023-11-17T00:06:31.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.740+0000 7f1f31ffb700 1 -- 172.21.15.6:0/2308737959 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f1f2809b0a0 con 0x7f1f34073550 2023-11-17T00:06:31.742 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:31.742 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:31.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.742+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 msgr2=0x7f1f240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.742+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 0x7f1f240a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f20005950 tx=0x7f1f20009260 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 msgr2=0x7f1f34198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f34198420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b070 tx=0x7f1f2800bab0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 shutdown_connections 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f240a11d0 0x7f1f240a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f34108190 0x7f1f3419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f34073550 0x7f1f34198420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 --2- 172.21.15.6:0/2308737959 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f34074b30 0x7f1f34198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 >> 172.21.15.6:0/2308737959 conn(0x7f1f340fb8b0 msgr2=0x7f1f340fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 shutdown_connections 2023-11-17T00:06:31.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.743+0000 7f1f3abff700 1 -- 172.21.15.6:0/2308737959 wait complete. 2023-11-17T00:06:31.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:06:31.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 -- 172.21.15.6:0/3615729393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c01002a0 msgr2=0x7f64c01006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 --2- 172.21.15.6:0/3615729393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c01002a0 0x7f64c01006b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009990 tx=0x7f64bc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 -- 172.21.15.6:0/3615729393 shutdown_connections 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 --2- 172.21.15.6:0/3615729393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c0101e20 0x7f64c01086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 --2- 172.21.15.6:0/3615729393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c01002a0 0x7f64c01006b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 --2- 172.21.15.6:0/3615729393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101490 0x7f64c01018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 -- 172.21.15.6:0/3615729393 >> 172.21.15.6:0/3615729393 conn(0x7f64c00fb880 msgr2=0x7f64c00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 -- 172.21.15.6:0/3615729393 shutdown_connections 2023-11-17T00:06:31.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.972+0000 7f64c73ff700 1 -- 172.21.15.6:0/3615729393 wait complete. 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.973+0000 7f64c73ff700 1 Processor -- start 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.973+0000 7f64c73ff700 1 -- start start 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 0x7f64c01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101e20 0x7f64c019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c019fd30 con 0x7f64c01002a0 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c019fe70 con 0x7f64c0101490 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c73ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c01a0000 con 0x7f64c0101e20 2023-11-17T00:06:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c519b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c519b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46414/0 (socket says 172.21.15.6:46414) 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c519b700 1 -- 172.21.15.6:0/1505388085 learned_addr learned my addr 172.21.15.6:0/1505388085 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c499a700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 0x7f64c01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.974+0000 7f64c599c700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101e20 0x7f64c019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 -- 172.21.15.6:0/1505388085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101e20 msgr2=0x7f64c019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101e20 0x7f64c019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 -- 172.21.15.6:0/1505388085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 msgr2=0x7f64c01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 0x7f64c01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 -- 172.21.15.6:0/1505388085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0095f0 con 0x7f64c01002a0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c499a700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 0x7f64c01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c519b700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64bc000c00 tx=0x7f64bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc01d070 con 0x7f64c01002a0 2023-11-17T00:06:31.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c010bc00 con 0x7f64c01002a0 2023-11-17T00:06:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64bc021410 con 0x7f64c01002a0 2023-11-17T00:06:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc017650 con 0x7f64c01002a0 2023-11-17T00:06:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.975+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c010ba70 con 0x7f64c01002a0 2023-11-17T00:06:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.976+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c01a0740 con 0x7f64c01002a0 2023-11-17T00:06:31.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.978+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f64bc021580 con 0x7f64c01002a0 2023-11-17T00:06:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.979+0000 7f64b67fc700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 0x7f64ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.980+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f64bc0cded0 con 0x7f64c01002a0 2023-11-17T00:06:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.980+0000 7f64c499a700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 0x7f64ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.981+0000 7f64c499a700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 0x7f64ac0a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64b0011490 tx=0x7f64b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:31.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:31.983+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f64bc097460 con 0x7f64c01002a0 2023-11-17T00:06:32.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:31 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1618729950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:32.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:31 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2308737959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:32.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:31 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1618729950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:32.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:31 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2308737959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:32.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.191+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64c01a0cf0 con 0x7f64c01002a0 2023-11-17T00:06:32.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.192+0000 7f64b67fc700 1 -- 172.21.15.6:0/1505388085 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64bc026020 con 0x7f64c01002a0 2023-11-17T00:06:32.194 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 msgr2=0x7f64ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 0x7f64ac0a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64b0011490 tx=0x7f64b0009260 comp rx=0 tx=0).stop 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 msgr2=0x7f64c01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64bc000c00 tx=0x7f64bc0057c0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 shutdown_connections 2023-11-17T00:06:32.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64ac0a1150 0x7f64ac0a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64c01002a0 0x7f64c01981b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64c0101e20 0x7f64c019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 --2- 172.21.15.6:0/1505388085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64c0101490 0x7f64c01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 >> 172.21.15.6:0/1505388085 conn(0x7f64c00fb880 msgr2=0x7f64c00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 shutdown_connections 2023-11-17T00:06:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.194+0000 7f64c73ff700 1 -- 172.21.15.6:0/1505388085 wait complete. 2023-11-17T00:06:32.220 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:31 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1618729950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:32.221 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:31 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2308737959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:32.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:06:32.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 -- 172.21.15.6:0/3461714134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104100290 msgr2=0x7f21041006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 --2- 172.21.15.6:0/3461714134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104100290 0x7f21041006a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f20f40098f0 tx=0x7f20f4009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 -- 172.21.15.6:0/3461714134 shutdown_connections 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 --2- 172.21.15.6:0/3461714134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104101e10 0x7f21041086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 --2- 172.21.15.6:0/3461714134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104100290 0x7f21041006a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 --2- 172.21.15.6:0/3461714134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f21041018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 -- 172.21.15.6:0/3461714134 >> 172.21.15.6:0/3461714134 conn(0x7f21040fb870 msgr2=0x7f21040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 -- 172.21.15.6:0/3461714134 shutdown_connections 2023-11-17T00:06:32.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.428+0000 7f2108bff700 1 -- 172.21.15.6:0/3461714134 wait complete. 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.429+0000 7f2108bff700 1 Processor -- start 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 -- start start 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 0x7f21041981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104101e10 0x7f210419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210419fd50 con 0x7f2104101480 2023-11-17T00:06:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210419fe90 con 0x7f2104100290 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2108bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041a0020 con 0x7f2104101e10 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46428/0 (socket says 172.21.15.6:46428) 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f20fbfff700 1 -- 172.21.15.6:0/2746488083 learned_addr learned my addr 172.21.15.6:0/2746488083 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f2102d9d700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104101e10 0x7f210419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.430+0000 7f210259c700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 0x7f21041981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 -- 172.21.15.6:0/2746488083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104101e10 msgr2=0x7f210419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104101e10 0x7f210419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 -- 172.21.15.6:0/2746488083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 msgr2=0x7f21041981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 0x7f21041981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 -- 172.21.15.6:0/2746488083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f4009520 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f210259c700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 0x7f21041981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fbfff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e9c0 tx=0x7f20ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004a30 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ec00cc80 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210410bbe0 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0054e0 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.431+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210410ba50 con 0x7f2104101480 2023-11-17T00:06:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.432+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104192870 con 0x7f2104101480 2023-11-17T00:06:32.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.434+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f20ec004b90 con 0x7f2104101480 2023-11-17T00:06:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.436+0000 7f20fb7fe700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 0x7f20f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.436+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f20ec014070 con 0x7f2104101480 2023-11-17T00:06:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.436+0000 7f210259c700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 0x7f20f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.437+0000 7f210259c700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 0x7f20f00a3680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009fd0 tx=0x7f20f4019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:32.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.440+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f20ec096080 con 0x7f2104101480 2023-11-17T00:06:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.647+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2104061700 con 0x7f2104101480 2023-11-17T00:06:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.648+0000 7f20fb7fe700 1 -- 172.21.15.6:0/2746488083 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20ec0996a0 con 0x7f2104101480 2023-11-17T00:06:32.650 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:06:32.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 msgr2=0x7f20f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 0x7f20f00a3680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009fd0 tx=0x7f20f4019040 comp rx=0 tx=0).stop 2023-11-17T00:06:32.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 msgr2=0x7f2104198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e9c0 tx=0x7f20ec00ecd0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 shutdown_connections 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f20f00a11d0 0x7f20f00a3680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2104101e10 0x7f210419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2104100290 0x7f21041981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 --2- 172.21.15.6:0/2746488083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2104101480 0x7f2104198710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.650+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 >> 172.21.15.6:0/2746488083 conn(0x7f21040fb870 msgr2=0x7f2104106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.651+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 shutdown_connections 2023-11-17T00:06:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.651+0000 7f2108bff700 1 -- 172.21.15.6:0/2746488083 wait complete. 2023-11-17T00:06:32.671 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:06:32.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 -- 172.21.15.6:0/1520264577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 msgr2=0x7f69c41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/1520264577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c41006a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009990 tx=0x7f69b8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 -- 172.21.15.6:0/1520264577 shutdown_connections 2023-11-17T00:06:32.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/1520264577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c41006a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/1520264577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101e10 0x7f69c41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/1520264577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101480 0x7f69c41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 -- 172.21.15.6:0/1520264577 >> 172.21.15.6:0/1520264577 conn(0x7f69c40fb870 msgr2=0x7f69c40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 -- 172.21.15.6:0/1520264577 shutdown_connections 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.879+0000 7f69cb1ff700 1 -- 172.21.15.6:0/1520264577 wait complete. 2023-11-17T00:06:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 Processor -- start 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 -- start start 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101480 0x7f69c41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101e10 0x7f69c419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.880+0000 7f69cb1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c419db30 con 0x7f69c4100290 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69cb1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c419dc70 con 0x7f69c4101480 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69cb1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c419de00 con 0x7f69c4101e10 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46452/0 (socket says 172.21.15.6:46452) 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 -- 172.21.15.6:0/411674914 learned_addr learned my addr 172.21.15.6:0/411674914 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:32.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c979c700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101e10 0x7f69c419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c3fff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101480 0x7f69c41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 -- 172.21.15.6:0/411674914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101e10 msgr2=0x7f69c419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101e10 0x7f69c419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 -- 172.21.15.6:0/411674914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101480 msgr2=0x7f69c41964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101480 0x7f69c41964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.881+0000 7f69c8f9b700 1 -- 172.21.15.6:0/411674914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b80095f0 con 0x7f69c4100290 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69c8f9b700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69b8000c00 tx=0x7f69b8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b801d070 con 0x7f69c4100290 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69b8022470 con 0x7f69c4100290 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c410bbf0 con 0x7f69c4100290 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8017630 con 0x7f69c4100290 2023-11-17T00:06:32.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.882+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c410ba60 con 0x7f69c4100290 2023-11-17T00:06:32.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.883+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c419e400 con 0x7f69c4100290 2023-11-17T00:06:32.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.885+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f69b80225e0 con 0x7f69c4100290 2023-11-17T00:06:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.887+0000 7f69c1ffb700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 0x7f69b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.887+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f69b80cde20 con 0x7f69c4100290 2023-11-17T00:06:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.888+0000 7f69c3fff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 0x7f69b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.888+0000 7f69c3fff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 0x7f69b40a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009c60 tx=0x7f69b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:32.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:32.891+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f69b805c730 con 0x7f69c4100290 2023-11-17T00:06:32.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:32 smithi006 ceph-mon[97840]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:32.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:32 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1505388085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:32.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:32 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2746488083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:32 smithi087 ceph-mon[131182]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:32 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1505388085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:32 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2746488083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.103+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69c4107200 con 0x7f69c4100290 2023-11-17T00:06:33.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.103+0000 7f69c1ffb700 1 -- 172.21.15.6:0/411674914 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69b80270a0 con 0x7f69c4100290 2023-11-17T00:06:33.105 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:06:33.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 msgr2=0x7f69b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 0x7f69b40a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009c60 tx=0x7f69b0009260 comp rx=0 tx=0).stop 2023-11-17T00:06:33.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 msgr2=0x7f69c4195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69b8000c00 tx=0x7f69b8005740 comp rx=0 tx=0).stop 2023-11-17T00:06:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 shutdown_connections 2023-11-17T00:06:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f69b40a11d0 0x7f69b40a3680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69c4100290 0x7f69c4195fb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69c4101e10 0x7f69c419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 --2- 172.21.15.6:0/411674914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69c4101480 0x7f69c41964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 >> 172.21.15.6:0/411674914 conn(0x7f69c40fb870 msgr2=0x7f69c40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:33.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 shutdown_connections 2023-11-17T00:06:33.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.106+0000 7f69cb1ff700 1 -- 172.21.15.6:0/411674914 wait complete. 2023-11-17T00:06:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:32 smithi186 ceph-mon[131487]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:32 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1505388085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:32 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2746488083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.128 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:06:33.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 -- 172.21.15.6:0/2504617367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f392810c860 msgr2=0x7f392810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 --2- 172.21.15.6:0/2504617367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f392810c860 0x7f392810ec40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f392000b0a0 tx=0x7f392000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 -- 172.21.15.6:0/2504617367 shutdown_connections 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 --2- 172.21.15.6:0/2504617367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f392810c860 0x7f392810ec40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 --2- 172.21.15.6:0/2504617367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3928074b30 0x7f3928072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 --2- 172.21.15.6:0/2504617367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928073550 0x7f39280739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.334+0000 7f392fdff700 1 -- 172.21.15.6:0/2504617367 >> 172.21.15.6:0/2504617367 conn(0x7f39280fb870 msgr2=0x7f39280fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.335+0000 7f392fdff700 1 -- 172.21.15.6:0/2504617367 shutdown_connections 2023-11-17T00:06:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.335+0000 7f392fdff700 1 -- 172.21.15.6:0/2504617367 wait complete. 2023-11-17T00:06:33.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 Processor -- start 2023-11-17T00:06:33.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 -- start start 2023-11-17T00:06:33.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928074b30 0x7f392819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 0x7f39281a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.336+0000 7f392fdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281a4250 con 0x7f3928073550 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392fdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281a4390 con 0x7f3928074b30 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281a4520 con 0x7f392810c860 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46474/0 (socket says 172.21.15.6:46474) 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 -- 172.21.15.6:0/597158361 learned_addr learned my addr 172.21.15.6:0/597158361 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392d39a700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928074b30 0x7f392819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392e39c700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 0x7f39281a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 -- 172.21.15.6:0/597158361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 msgr2=0x7f39281a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 0x7f39281a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 -- 172.21.15.6:0/597158361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928074b30 msgr2=0x7f392819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928074b30 0x7f392819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 -- 172.21.15.6:0/597158361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009520 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392e39c700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 0x7f39281a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.337+0000 7f392db9b700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f392400e970 tx=0x7f392400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.338+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924009790 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.338+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392400eec0 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.338+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3920009d00 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.338+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924005590 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.338+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3928112270 con 0x7f3928073550 2023-11-17T00:06:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.339+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39281023b0 con 0x7f3928073550 2023-11-17T00:06:33.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.341+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f392400cbd0 con 0x7f3928073550 2023-11-17T00:06:33.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.342+0000 7f391effd700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 0x7f39140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.342+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3924014070 con 0x7f3928073550 2023-11-17T00:06:33.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.343+0000 7f392d39a700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 0x7f39140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.343+0000 7f392d39a700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 0x7f39140a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3928074850 tx=0x7f3918009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:33.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.344+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3924096480 con 0x7f3928073550 2023-11-17T00:06:33.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.554+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f392802cfe0 con 0x7f3928073550 2023-11-17T00:06:33.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.554+0000 7f391effd700 1 -- 172.21.15.6:0/597158361 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3924004ef0 con 0x7f3928073550 2023-11-17T00:06:33.557 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:06:33.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 msgr2=0x7f39140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 0x7f39140a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3928074850 tx=0x7f3918009260 comp rx=0 tx=0).stop 2023-11-17T00:06:33.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 msgr2=0x7f392819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f392400e970 tx=0x7f392400ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:33.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 shutdown_connections 2023-11-17T00:06:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f39140a1150 0x7f39140a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3928073550 0x7f392819c6d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f392810c860 0x7f39281a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 --2- 172.21.15.6:0/597158361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3928074b30 0x7f392819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 >> 172.21.15.6:0/597158361 conn(0x7f39280fb870 msgr2=0x7f39280fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:33.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 shutdown_connections 2023-11-17T00:06:33.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.557+0000 7f392fdff700 1 -- 172.21.15.6:0/597158361 wait complete. 2023-11-17T00:06:33.578 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-17T00:06:33.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 -- 172.21.15.6:0/3545564417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 msgr2=0x7f5e38108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/3545564417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38108670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b0a0 tx=0x7f5e3000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 -- 172.21.15.6:0/3545564417 shutdown_connections 2023-11-17T00:06:33.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/3545564417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38108670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/3545564417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38100230 0x7f5e38100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/3545564417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38101420 0x7f5e38101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 -- 172.21.15.6:0/3545564417 >> 172.21.15.6:0/3545564417 conn(0x7f5e380fb810 msgr2=0x7f5e380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 -- 172.21.15.6:0/3545564417 shutdown_connections 2023-11-17T00:06:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.784+0000 7f5e3ddff700 1 -- 172.21.15.6:0/3545564417 wait complete. 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.785+0000 7f5e3ddff700 1 Processor -- start 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.785+0000 7f5e3ddff700 1 -- start start 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38101420 0x7f5e38072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38073bc0 con 0x7f5e38101db0 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38073d30 con 0x7f5e38100230 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e3ddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e381a0100 con 0x7f5e38101420 2023-11-17T00:06:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e377fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e36ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38101420 0x7f5e38072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e377fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55612/0 (socket says 172.21.15.6:55612) 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.786+0000 7f5e377fe700 1 -- 172.21.15.6:0/2684011934 learned_addr learned my addr 172.21.15.6:0/2684011934 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 -- 172.21.15.6:0/2684011934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38101420 msgr2=0x7f5e38072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38101420 0x7f5e38072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 -- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 msgr2=0x7f5e38073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 -- 172.21.15.6:0/2684011934 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28009520 con 0x7f5e38100230 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e37fff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e377fe700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800ea50 tx=0x7f5e2800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cbd0 con 0x7f5e38100230 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c009520 con 0x7f5e38100230 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2800cd30 con 0x7f5e38100230 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28018680 con 0x7f5e38100230 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.787+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e30009d00 con 0x7f5e38100230 2023-11-17T00:06:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.788+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3810ba70 con 0x7f5e38100230 2023-11-17T00:06:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.790+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5e28010430 con 0x7f5e38100230 2023-11-17T00:06:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.792+0000 7f5e34ff9700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 0x7f5e200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.792+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5e28014070 con 0x7f5e38100230 2023-11-17T00:06:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.792+0000 7f5e36ffd700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 0x7f5e200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.793+0000 7f5e36ffd700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 0x7f5e200a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009df0 tx=0x7f5e2c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:33.794+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e280969b0 con 0x7f5e38100230 2023-11-17T00:06:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:33 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/411674914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:33.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:33 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/597158361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:34.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.019+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e38107090 con 0x7f5e38100230 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.019+0000 7f5e34ff9700 1 -- 172.21.15.6:0/2684011934 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+580 (secure 0 0 0) 0x7f5e28099fd0 con 0x7f5e38100230 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1821215069 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4157530954 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3407461840 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3069406217 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:34.021 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:34.022 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:34.022 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3467557048 2023-11-17T23:58:33.059992+0000 2023-11-17T00:06:34.022 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2971625275 2023-11-17T23:59:01.653365+0000 2023-11-17T00:06:34.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 msgr2=0x7f5e200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 0x7f5e200a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009df0 tx=0x7f5e2c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 msgr2=0x7f5e380748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800ea50 tx=0x7f5e2800ed60 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 shutdown_connections 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5e200a1150 0x7f5e200a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e38101db0 0x7f5e38073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e38100230 0x7f5e380748a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 --2- 172.21.15.6:0/2684011934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e38101420 0x7f5e38072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 >> 172.21.15.6:0/2684011934 conn(0x7f5e380fb810 msgr2=0x7f5e380fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 shutdown_connections 2023-11-17T00:06:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.022+0000 7f5e3ddff700 1 -- 172.21.15.6:0/2684011934 wait complete. 2023-11-17T00:06:34.025 INFO:teuthology.orchestra.run.smithi006.stderr:listed 10 entries 2023-11-17T00:06:34.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-17T00:06:34.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:34.043 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-17T00:06:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:33 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/411674914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:34.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:33 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/597158361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:34.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[134438]: fuse finished with error 0 and tester_r 0 2023-11-17T00:06:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:33 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/411674914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:06:34.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:33 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/597158361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:06:34.116 INFO:teuthology.orchestra.run:waiting for 300 2023-11-17T00:06:34.196 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:06:34.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:06:34.196 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-17T00:06:34.250 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 -- 172.21.15.6:0/1840446798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc1002d0 msgr2=0x7f30fc1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 --2- 172.21.15.6:0/1840446798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc1002d0 0x7f30fc1006e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009990 tx=0x7f30ec009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 -- 172.21.15.6:0/1840446798 shutdown_connections 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 --2- 172.21.15.6:0/1840446798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc101e50 0x7f30fc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 --2- 172.21.15.6:0/1840446798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc1002d0 0x7f30fc1006e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 --2- 172.21.15.6:0/1840446798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 -- 172.21.15.6:0/1840446798 >> 172.21.15.6:0/1840446798 conn(0x7f30fc0fb8b0 msgr2=0x7f30fc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:34.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 -- 172.21.15.6:0/1840446798 shutdown_connections 2023-11-17T00:06:34.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.454+0000 7f31021ff700 1 -- 172.21.15.6:0/1840446798 wait complete. 2023-11-17T00:06:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.455+0000 7f31021ff700 1 Processor -- start 2023-11-17T00:06:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.455+0000 7f31021ff700 1 -- start start 2023-11-17T00:06:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 0x7f30fc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc101e50 0x7f30fc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19fdd0 con 0x7f30fc1014c0 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19ff10 con 0x7f30fc101e50 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f31021ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc1a00a0 con 0x7f30fc1002d0 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30fb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 0x7f30fc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46500/0 (socket says 172.21.15.6:46500) 2023-11-17T00:06:34.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 -- 172.21.15.6:0/2381555444 learned_addr learned my addr 172.21.15.6:0/2381555444 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:34.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30fbfff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc101e50 0x7f30fc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 -- 172.21.15.6:0/2381555444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 msgr2=0x7f30fc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 0x7f30fc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 -- 172.21.15.6:0/2381555444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc101e50 msgr2=0x7f30fc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc101e50 0x7f30fc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30faffd700 1 -- 172.21.15.6:0/2381555444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0095f0 con 0x7f30fc1014c0 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.456+0000 7f30fb7fe700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 0x7f30fc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f30faffd700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f30f000e9a0 tx=0x7f30f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0009790 con 0x7f30fc1014c0 2023-11-17T00:06:34.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc10bc30 con 0x7f30fc1014c0 2023-11-17T00:06:34.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f0010430 con 0x7f30fc1014c0 2023-11-17T00:06:34.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc10baa0 con 0x7f30fc1014c0 2023-11-17T00:06:34.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.457+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0018620 con 0x7f30fc1014c0 2023-11-17T00:06:34.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.458+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc1a07e0 con 0x7f30fc1014c0 2023-11-17T00:06:34.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.460+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f30f00105a0 con 0x7f30fc1014c0 2023-11-17T00:06:34.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.462+0000 7f30f8ff9700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 0x7f30e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.463+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f30f0014070 con 0x7f30fc1014c0 2023-11-17T00:06:34.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.463+0000 7f30fb7fe700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 0x7f30e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.463+0000 7f30fb7fe700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 0x7f30e40a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00b2e0 tx=0x7f30ec005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:34.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.467+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f30f0051730 con 0x7f30fc1014c0 2023-11-17T00:06:34.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.676+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30fc1a0e80 con 0x7f30fc1014c0 2023-11-17T00:06:34.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.677+0000 7f30f8ff9700 1 -- 172.21.15.6:0/2381555444 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f30f00961b0 con 0x7f30fc1014c0 2023-11-17T00:06:34.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:34.679 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:34.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 msgr2=0x7f30e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 0x7f30e40a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00b2e0 tx=0x7f30ec005960 comp rx=0 tx=0).stop 2023-11-17T00:06:34.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 msgr2=0x7f30fc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f30f000e9a0 tx=0x7f30f000ecb0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 shutdown_connections 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f30e40a1150 0x7f30e40a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30fc101e50 0x7f30fc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30fc1002d0 0x7f30fc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 --2- 172.21.15.6:0/2381555444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30fc1014c0 0x7f30fc198790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 >> 172.21.15.6:0/2381555444 conn(0x7f30fc0fb8b0 msgr2=0x7f30fc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:34.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 shutdown_connections 2023-11-17T00:06:34.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.680+0000 7f31021ff700 1 -- 172.21.15.6:0/2381555444 wait complete. 2023-11-17T00:06:34.701 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 -- 172.21.15.6:0/4053867899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 msgr2=0x7f9c081006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/4053867899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081006e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0098f0 tx=0x7f9bfc009c00 comp rx=0 tx=0).stop 2023-11-17T00:06:34.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 -- 172.21.15.6:0/4053867899 shutdown_connections 2023-11-17T00:06:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/4053867899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 0x7f9c08108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/4053867899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081006e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/4053867899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 0x7f9c08101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 -- 172.21.15.6:0/4053867899 >> 172.21.15.6:0/4053867899 conn(0x7f9c080fb8b0 msgr2=0x7f9c080fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 -- 172.21.15.6:0/4053867899 shutdown_connections 2023-11-17T00:06:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.907+0000 7f9c0ebff700 1 -- 172.21.15.6:0/4053867899 wait complete. 2023-11-17T00:06:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.908+0000 7f9c0ebff700 1 Processor -- start 2023-11-17T00:06:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 -- start start 2023-11-17T00:06:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 0x7f9c081987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 0x7f9c0819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819fe30 con 0x7f9c081014c0 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819ff70 con 0x7f9c081002d0 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081a0100 con 0x7f9c08101e50 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0c99b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c07fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 0x7f9c081987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0d19c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 0x7f9c0819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0c99b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55644/0 (socket says 172.21.15.6:55644) 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.909+0000 7f9c0c99b700 1 -- 172.21.15.6:0/2409405229 learned_addr learned my addr 172.21.15.6:0/2409405229 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 -- 172.21.15.6:0/2409405229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 msgr2=0x7f9c0819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 0x7f9c0819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 -- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 msgr2=0x7f9c081987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 0x7f9c081987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 -- 172.21.15.6:0/2409405229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc009520 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0c99b700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc01c070 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bfc00f7b0 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0810bc30 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.910+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0039b0 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.911+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0810baa0 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.912+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c081a0700 con 0x7f9c081002d0 2023-11-17T00:06:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.913+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9bfc020030 con 0x7f9c081002d0 2023-11-17T00:06:34.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.915+0000 7f9c05ffb700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 0x7f9bf80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:34.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.915+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9bfc0cdba0 con 0x7f9c081002d0 2023-11-17T00:06:34.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.915+0000 7f9c07fff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 0x7f9bf80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:34.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.916+0000 7f9c07fff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 0x7f9bf80a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005fd0 tx=0x7f9bf4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:34.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:34.919+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9bfc097130 con 0x7f9c081002d0 2023-11-17T00:06:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:34 smithi006 ceph-mon[97840]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:34.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:34 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2684011934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:34.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:34 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2381555444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:34 smithi087 ceph-mon[131182]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:34 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2684011934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:34 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2381555444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:34 smithi186 ceph-mon[131487]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:34 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2684011934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:06:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:34 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2381555444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:35.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.128+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c081a0cb0 con 0x7f9c081002d0 2023-11-17T00:06:35.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.131+0000 7f9c05ffb700 1 -- 172.21.15.6:0/2409405229 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f9bfc09a750 con 0x7f9c081002d0 2023-11-17T00:06:35.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:35.134 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:35.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 msgr2=0x7f9bf80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 0x7f9bf80a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005fd0 tx=0x7f9bf4009310 comp rx=0 tx=0).stop 2023-11-17T00:06:35.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 msgr2=0x7f9c081982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc000c00 tx=0x7f9bfc00b730 comp rx=0 tx=0).stop 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 shutdown_connections 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9bf80a11d0 0x7f9bf80a3680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08101e50 0x7f9c0819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c081002d0 0x7f9c081982b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 --2- 172.21.15.6:0/2409405229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c081014c0 0x7f9c081987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 >> 172.21.15.6:0/2409405229 conn(0x7f9c080fb8b0 msgr2=0x7f9c080fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 shutdown_connections 2023-11-17T00:06:35.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.133+0000 7f9c0ebff700 1 -- 172.21.15.6:0/2409405229 wait complete. 2023-11-17T00:06:35.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-17T00:06:35.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 -- 172.21.15.6:0/744378753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101480 msgr2=0x7ffa4c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 --2- 172.21.15.6:0/744378753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101480 0x7ffa4c1018d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009990 tx=0x7ffa40009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 -- 172.21.15.6:0/744378753 shutdown_connections 2023-11-17T00:06:35.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 --2- 172.21.15.6:0/744378753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c101e10 0x7ffa4c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 --2- 172.21.15.6:0/744378753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c100290 0x7ffa4c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 --2- 172.21.15.6:0/744378753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101480 0x7ffa4c1018d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 -- 172.21.15.6:0/744378753 >> 172.21.15.6:0/744378753 conn(0x7ffa4c0fb870 msgr2=0x7ffa4c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:35.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 -- 172.21.15.6:0/744378753 shutdown_connections 2023-11-17T00:06:35.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.359+0000 7ffa515ff700 1 -- 172.21.15.6:0/744378753 wait complete. 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.360+0000 7ffa515ff700 1 Processor -- start 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 -- start start 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c100290 0x7ffa4c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c101480 0x7ffa4c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19fd60 con 0x7ffa4c101e10 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19fea0 con 0x7ffa4c100290 2023-11-17T00:06:35.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa515ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1a0030 con 0x7ffa4c101480 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46530/0 (socket says 172.21.15.6:46530) 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa4b7fe700 1 -- 172.21.15.6:0/443379794 learned_addr learned my addr 172.21.15.6:0/443379794 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa4a7fc700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c101480 0x7ffa4c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.361+0000 7ffa4affd700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c100290 0x7ffa4c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 -- 172.21.15.6:0/443379794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c101480 msgr2=0x7ffa4c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c101480 0x7ffa4c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 -- 172.21.15.6:0/443379794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c100290 msgr2=0x7ffa4c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c100290 0x7ffa4c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 -- 172.21.15.6:0/443379794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009520 con 0x7ffa4c101e10 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa4b7fe700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ef30 tx=0x7ffa4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44018070 con 0x7ffa4c101e10 2023-11-17T00:06:35.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4400f040 con 0x7ffa4c101e10 2023-11-17T00:06:35.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa4c101e10 2023-11-17T00:06:35.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400cd00 con 0x7ffa4c101e10 2023-11-17T00:06:35.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.362+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c10bc50 con 0x7ffa4c101e10 2023-11-17T00:06:35.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.363+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c10bde0 con 0x7ffa4c101e10 2023-11-17T00:06:35.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.365+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffa44003a00 con 0x7ffa4c101e10 2023-11-17T00:06:35.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.367+0000 7ffa33fff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 0x7ffa340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:35.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.368+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffa440cc680 con 0x7ffa4c101e10 2023-11-17T00:06:35.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.368+0000 7ffa4affd700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 0x7ffa340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:35.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.368+0000 7ffa4affd700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 0x7ffa340a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c1012e0 tx=0x7ffa3c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:35.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.371+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffa44095c10 con 0x7ffa4c101e10 2023-11-17T00:06:35.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.628+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffa4c067a50 con 0x7ffa4c101e10 2023-11-17T00:06:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.828+0000 7ffa33fff700 1 -- 172.21.15.6:0/443379794 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffa44099230 con 0x7ffa4c101e10 2023-11-17T00:06:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 msgr2=0x7ffa340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 0x7ffa340a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c1012e0 tx=0x7ffa3c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 msgr2=0x7ffa4c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ef30 tx=0x7ffa4400c3a0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 shutdown_connections 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa340a1150 0x7ffa340a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa4c101e10 0x7ffa4c19f790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa4c100290 0x7ffa4c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 --2- 172.21.15.6:0/443379794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa4c101480 0x7ffa4c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 >> 172.21.15.6:0/443379794 conn(0x7ffa4c0fb870 msgr2=0x7ffa4c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 shutdown_connections 2023-11-17T00:06:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:35.831+0000 7ffa515ff700 1 -- 172.21.15.6:0/443379794 wait complete. 2023-11-17T00:06:35.851 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-11-17T00:06:35.851 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-17T00:06:36.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 -- 172.21.15.6:0/2039981697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990101480 msgr2=0x7f29901018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:36.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 --2- 172.21.15.6:0/2039981697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990101480 0x7f29901018d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2980009990 tx=0x7f2980009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 -- 172.21.15.6:0/2039981697 shutdown_connections 2023-11-17T00:06:36.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 --2- 172.21.15.6:0/2039981697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 0x7f29901086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 --2- 172.21.15.6:0/2039981697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990100290 0x7f29901006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 --2- 172.21.15.6:0/2039981697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990101480 0x7f29901018d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 -- 172.21.15.6:0/2039981697 >> 172.21.15.6:0/2039981697 conn(0x7f29900fb870 msgr2=0x7f29900fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:36.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 -- 172.21.15.6:0/2039981697 shutdown_connections 2023-11-17T00:06:36.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.058+0000 7f29975ff700 1 -- 172.21.15.6:0/2039981697 wait complete. 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.059+0000 7f29975ff700 1 Processor -- start 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 -- start start 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 0x7f2990198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 0x7f299019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019fd60 con 0x7f2990100290 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019fea0 con 0x7f2990101480 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f29975ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29901a0030 con 0x7f2990101e10 2023-11-17T00:06:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f299539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f299539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46550/0 (socket says 172.21.15.6:46550) 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f299539b700 1 -- 172.21.15.6:0/2341981878 learned_addr learned my addr 172.21.15.6:0/2341981878 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f2994b9a700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 0x7f2990198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.060+0000 7f2995b9c700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 0x7f299019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 -- 172.21.15.6:0/2341981878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 msgr2=0x7f299019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 0x7f299019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 -- 172.21.15.6:0/2341981878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 msgr2=0x7f2990198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 0x7f2990198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 -- 172.21.15.6:0/2341981878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800095f0 con 0x7f2990100290 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f2994b9a700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 0x7f2990198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f299539b700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e970 tx=0x7f298c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c009790 con 0x7f2990100290 2023-11-17T00:06:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f298c00c7e0 con 0x7f2990100290 2023-11-17T00:06:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299010bc50 con 0x7f2990100290 2023-11-17T00:06:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c003c70 con 0x7f2990100290 2023-11-17T00:06:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.061+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299010bac0 con 0x7f2990100290 2023-11-17T00:06:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.063+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29901a07d0 con 0x7f2990100290 2023-11-17T00:06:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.064+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f298c00cc50 con 0x7f2990100290 2023-11-17T00:06:36.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.067+0000 7f29867fc700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 0x7f297c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:36.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.067+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f298c014070 con 0x7f2990100290 2023-11-17T00:06:36.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.067+0000 7f2994b9a700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 0x7f297c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:36.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.068+0000 7f2994b9a700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 0x7f297c0a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f298000b2e0 tx=0x7f2980005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:36.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.071+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f298c0961a0 con 0x7f2990100290 2023-11-17T00:06:36.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:35 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2409405229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:36.106 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:35 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-17T00:06:36.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:35 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2409405229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:36.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:35 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-17T00:06:36.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:35 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2409405229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:36.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:35 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-17T00:06:36.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.328+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f29901a0d80 con 0x7f2990100290 2023-11-17T00:06:36.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.830+0000 7f29867fc700 1 -- 172.21.15.6:0/2341981878 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f298c003dd0 con 0x7f2990100290 2023-11-17T00:06:36.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 msgr2=0x7f297c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:36.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 0x7f297c0a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f298000b2e0 tx=0x7f2980005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 msgr2=0x7f29901981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:36.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e970 tx=0x7f298c00ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:36.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 shutdown_connections 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f297c0a11d0 0x7f297c0a3680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2990100290 0x7f29901981e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2990101e10 0x7f299019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.833+0000 7f29975ff700 1 --2- 172.21.15.6:0/2341981878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2990101480 0x7f2990198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.834+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 >> 172.21.15.6:0/2341981878 conn(0x7f29900fb870 msgr2=0x7f29900fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.834+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 shutdown_connections 2023-11-17T00:06:36.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:36.834+0000 7f29975ff700 1 -- 172.21.15.6:0/2341981878 wait complete. 2023-11-17T00:06:36.856 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 -- 172.21.15.6:0/1248345458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464100290 msgr2=0x7f34641006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 --2- 172.21.15.6:0/1248345458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464100290 0x7f34641006a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3460009990 tx=0x7f3460009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 -- 172.21.15.6:0/1248345458 shutdown_connections 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 --2- 172.21.15.6:0/1248345458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f34641086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 --2- 172.21.15.6:0/1248345458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464100290 0x7f34641006a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 --2- 172.21.15.6:0/1248345458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464101480 0x7f34641018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 -- 172.21.15.6:0/1248345458 >> 172.21.15.6:0/1248345458 conn(0x7f34640fb870 msgr2=0x7f34640fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 -- 172.21.15.6:0/1248345458 shutdown_connections 2023-11-17T00:06:37.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.064+0000 7f346bfff700 1 -- 172.21.15.6:0/1248345458 wait complete. 2023-11-17T00:06:37.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 Processor -- start 2023-11-17T00:06:37.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 -- start start 2023-11-17T00:06:37.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464101480 0x7f34641986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419fd10 con 0x7f3464101e10 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419fe50 con 0x7f3464100290 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.065+0000 7f346bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419ffe0 con 0x7f3464101480 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f346a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55704/0 (socket says 172.21.15.6:55704) 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 -- 172.21.15.6:0/959373510 learned_addr learned my addr 172.21.15.6:0/959373510 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f346a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46568/0 (socket says 172.21.15.6:46568) 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f346959a700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464101480 0x7f34641986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 -- 172.21.15.6:0/959373510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464101480 msgr2=0x7f34641986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:37.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464101480 0x7f34641986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 -- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 msgr2=0x7f346419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 -- 172.21.15.6:0/959373510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600095f0 con 0x7f3464100290 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f346a59c700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.066+0000 7f3469d9b700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3460005720 tx=0x7f3460009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.067+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346001d070 con 0x7f3464100290 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.067+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3460021410 con 0x7f3464100290 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.067+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460017650 con 0x7f3464100290 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.067+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346410bbe0 con 0x7f3464100290 2023-11-17T00:06:37.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.067+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346410ba50 con 0x7f3464100290 2023-11-17T00:06:37.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.068+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464067a50 con 0x7f3464100290 2023-11-17T00:06:37.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.070+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3460021580 con 0x7f3464100290 2023-11-17T00:06:37.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.071+0000 7f345affd700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 0x7f34500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:37.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.071+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34600ce480 con 0x7f3464100290 2023-11-17T00:06:37.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.072+0000 7f346959a700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 0x7f34500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:37.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.072+0000 7f346959a700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 0x7f34500a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3454009c60 tx=0x7f3454009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:37.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.073+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3460097a10 con 0x7f3464100290 2023-11-17T00:06:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:36 smithi087 ceph-mon[131182]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:36 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-17T00:06:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:36 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:36 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-17T00:06:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:36 smithi186 ceph-mon[131487]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:36 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-17T00:06:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:36 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:37.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:36 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-17T00:06:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:36 smithi006 ceph-mon[97840]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-17T00:06:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:36 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/443379794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-17T00:06:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:36 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:36 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-17T00:06:37.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.334+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f34641a09b0 con 0x7f3464100290 2023-11-17T00:06:37.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.841+0000 7f345affd700 1 -- 172.21.15.6:0/959373510 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f346009b030 con 0x7f3464100290 2023-11-17T00:06:37.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 msgr2=0x7f34500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:37.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 0x7f34500a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3454009c60 tx=0x7f3454009260 comp rx=0 tx=0).stop 2023-11-17T00:06:37.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 msgr2=0x7f3464198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:37.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3460005720 tx=0x7f3460009580 comp rx=0 tx=0).stop 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 shutdown_connections 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f34500a11d0 0x7f34500a3680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3464101e10 0x7f346419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3464100290 0x7f3464198190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 --2- 172.21.15.6:0/959373510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464101480 0x7f34641986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.843+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 >> 172.21.15.6:0/959373510 conn(0x7f34640fb870 msgr2=0x7f34640fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.844+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 shutdown_connections 2023-11-17T00:06:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:37.844+0000 7f346bfff700 1 -- 172.21.15.6:0/959373510 wait complete. 2023-11-17T00:06:37.865 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-11-17T00:06:37.865 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 -- 172.21.15.6:0/3175280798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c105fd0 msgr2=0x7f662c1083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 --2- 172.21.15.6:0/3175280798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c105fd0 0x7f662c1083b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f662400b0a0 tx=0x7f662400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 -- 172.21.15.6:0/3175280798 shutdown_connections 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 --2- 172.21.15.6:0/3175280798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c105fd0 0x7f662c1083b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 --2- 172.21.15.6:0/3175280798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c100d90 0x7f662c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 --2- 172.21.15.6:0/3175280798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c1036b0 0x7f662c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 -- 172.21.15.6:0/3175280798 >> 172.21.15.6:0/3175280798 conn(0x7f662c0fa750 msgr2=0x7f662c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 -- 172.21.15.6:0/3175280798 shutdown_connections 2023-11-17T00:06:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.074+0000 7f66343ff700 1 -- 172.21.15.6:0/3175280798 wait complete. 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.075+0000 7f66343ff700 1 Processor -- start 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.075+0000 7f66343ff700 1 -- start start 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c1036b0 0x7f662c0fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 0x7f662c0ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0ffa20 con 0x7f662c100d90 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0ffb90 con 0x7f662c105fd0 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f66343ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c1a0160 con 0x7f662c1036b0 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46586/0 (socket says 172.21.15.6:46586) 2023-11-17T00:06:38.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 -- 172.21.15.6:0/211338992 learned_addr learned my addr 172.21.15.6:0/211338992 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663199a700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c1036b0 0x7f662c0fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663299c700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 0x7f662c0ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 -- 172.21.15.6:0/211338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c1036b0 msgr2=0x7f662c0fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c1036b0 0x7f662c0fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 -- 172.21.15.6:0/211338992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 msgr2=0x7f662c0ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 0x7f662c0ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663219b700 1 -- 172.21.15.6:0/211338992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624009d00 con 0x7f662c100d90 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.076+0000 7f663299c700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 0x7f662c0ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f663219b700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f662800e970 tx=0x7f662800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628009790 con 0x7f662c100d90 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662800eec0 con 0x7f662c100d90 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c10bc60 con 0x7f662c100d90 2023-11-17T00:06:38.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628005590 con 0x7f662c100d90 2023-11-17T00:06:38.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.077+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c10bad0 con 0x7f662c100d90 2023-11-17T00:06:38.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.078+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c0fc2c0 con 0x7f662c100d90 2023-11-17T00:06:38.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.080+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f662800cbd0 con 0x7f662c100d90 2023-11-17T00:06:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.081+0000 7f66237fe700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 0x7f66180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.081+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6628014070 con 0x7f662c100d90 2023-11-17T00:06:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.081+0000 7f663199a700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 0x7f66180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.082+0000 7f663199a700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 0x7f66180a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f661c009c60 tx=0x7f661c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:38.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.085+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f66280962a0 con 0x7f662c100d90 2023-11-17T00:06:38.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-17T00:06:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:37 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:37 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/959373510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:37 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-17T00:06:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:37 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:37 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/959373510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:37 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2341981878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-17T00:06:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:37 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-17T00:06:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:37 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/959373510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:37 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-17T00:06:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.329+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f662c061c40 con 0x7f66180a1150 2023-11-17T00:06:38.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.368+0000 7f66237fe700 1 -- 172.21.15.6:0/211338992 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f662c061c40 con 0x7f66180a1150 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 msgr2=0x7f66180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 0x7f66180a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f661c009c60 tx=0x7f661c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 msgr2=0x7f662c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f662800e970 tx=0x7f662800ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 shutdown_connections 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f66180a1150 0x7f66180a3600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f662c100d90 0x7f662c100700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f662c105fd0 0x7f662c0ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 --2- 172.21.15.6:0/211338992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f662c1036b0 0x7f662c0fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 >> 172.21.15.6:0/211338992 conn(0x7f662c0fa750 msgr2=0x7f662c101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:38.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 shutdown_connections 2023-11-17T00:06:38.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.370+0000 7f66343ff700 1 -- 172.21.15.6:0/211338992 wait complete. 2023-11-17T00:06:38.391 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 -- 172.21.15.6:0/23503024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a40734d0 msgr2=0x7ff4a4073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 --2- 172.21.15.6:0/23503024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a40734d0 0x7ff4a4073940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff494009990 tx=0x7ff494009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 -- 172.21.15.6:0/23503024 shutdown_connections 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 --2- 172.21.15.6:0/23503024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a4108190 0x7ff4a410a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 --2- 172.21.15.6:0/23503024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 0x7ff4a4072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 --2- 172.21.15.6:0/23503024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a40734d0 0x7ff4a4073940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 -- 172.21.15.6:0/23503024 >> 172.21.15.6:0/23503024 conn(0x7ff4a40fb900 msgr2=0x7ff4a40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 -- 172.21.15.6:0/23503024 shutdown_connections 2023-11-17T00:06:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.597+0000 7ff4abbff700 1 -- 172.21.15.6:0/23503024 wait complete. 2023-11-17T00:06:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.598+0000 7ff4abbff700 1 Processor -- start 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.598+0000 7ff4abbff700 1 -- start start 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 0x7ff4a41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 0x7ff4a4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419fd60 con 0x7ff4a4108190 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419fea0 con 0x7ff4a40734d0 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4abbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41a0030 con 0x7ff4a4074ab0 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46620/0 (socket says 172.21.15.6:46620) 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4a999b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 0x7ff4a41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 -- 172.21.15.6:0/1803112190 learned_addr learned my addr 172.21.15.6:0/1803112190 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4a919a700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 0x7ff4a4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 -- 172.21.15.6:0/1803112190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 msgr2=0x7ff4a4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 0x7ff4a4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 -- 172.21.15.6:0/1803112190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 msgr2=0x7ff4a41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 0x7ff4a41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4aa19c700 1 -- 172.21.15.6:0/1803112190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c009c40 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.599+0000 7ff4a999b700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 0x7ff4a41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff4aa19c700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c370 tx=0x7ff49c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c017070 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c00f040 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00e050 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4940095f0 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.600+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a410de80 con 0x7ff4a4108190 2023-11-17T00:06:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.601+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a41068d0 con 0x7ff4a4108190 2023-11-17T00:06:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.603+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff49c00cb60 con 0x7ff4a4108190 2023-11-17T00:06:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.605+0000 7ff49affd700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.605+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff49c0ccad0 con 0x7ff4a4108190 2023-11-17T00:06:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.605+0000 7ff4a999b700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.606+0000 7ff4a999b700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 0x7ff4900a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0011490 tx=0x7ff4a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:38.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.607+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff49c095fd0 con 0x7ff4a4108190 2023-11-17T00:06:38.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:06:38.336+0000 7fdb75f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:06:38.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:06:38.336+0000 7fdb75f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:06:38.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:06:38.336+0000 7fdb75f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:06:38.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:06:38.336+0000 7fdb75f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:06:38.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:06:38.336+0000 7fdb75f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:06:38.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.853+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4a4061980 con 0x7ff4900a11d0 2023-11-17T00:06:38.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.857+0000 7ff49affd700 1 -- 172.21.15.6:0/1803112190 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7ff4a4061980 con 0x7ff4900a11d0 2023-11-17T00:06:38.858 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/_nogroup/sv_0/fad35ed3-1ec7-47f4-aa67-8233add0af97 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 msgr2=0x7ff4900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 0x7ff4900a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0011490 tx=0x7ff4a0009260 comp rx=0 tx=0).stop 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 msgr2=0x7ff4a419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c370 tx=0x7ff49c00c680 comp rx=0 tx=0).stop 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 shutdown_connections 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4a4108190 0x7ff4a419f790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a40734d0 0x7ff4a41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 --2- 172.21.15.6:0/1803112190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a4074ab0 0x7ff4a4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:38.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 >> 172.21.15.6:0/1803112190 conn(0x7ff4a40fb900 msgr2=0x7ff4a40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:38.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 shutdown_connections 2023-11-17T00:06:38.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:38.859+0000 7ff4abbff700 1 -- 172.21.15.6:0/1803112190 wait complete. 2023-11-17T00:06:38.880 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 -- 172.21.15.6:0/2324205772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e60074b30 msgr2=0x7f1e60072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 --2- 172.21.15.6:0/2324205772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e60074b30 0x7f1e60072f80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009990 tx=0x7f1e50009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 -- 172.21.15.6:0/2324205772 shutdown_connections 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 --2- 172.21.15.6:0/2324205772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e60074b30 0x7f1e60072f80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 --2- 172.21.15.6:0/2324205772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6010c860 0x7f1e6010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 --2- 172.21.15.6:0/2324205772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60073550 0x7f1e600739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 -- 172.21.15.6:0/2324205772 >> 172.21.15.6:0/2324205772 conn(0x7f1e600fb870 msgr2=0x7f1e600fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:39.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 -- 172.21.15.6:0/2324205772 shutdown_connections 2023-11-17T00:06:39.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.085+0000 7f1e65fff700 1 -- 172.21.15.6:0/2324205772 wait complete. 2023-11-17T00:06:39.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 Processor -- start 2023-11-17T00:06:39.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 -- start start 2023-11-17T00:06:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e60073550 0x7f1e6019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e6010c860 0x7f1e601a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601a4170 con 0x7f1e6010c860 2023-11-17T00:06:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601a42b0 con 0x7f1e60073550 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.087+0000 7f1e65fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601a4440 con 0x7f1e60074b30 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e6010c860 0x7f1e601a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:39572/0 (socket says 172.21.15.6:39572) 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 -- 172.21.15.6:0/3529524062 learned_addr learned my addr 172.21.15.6:0/3529524062 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:39.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5f7fe700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e60073550 0x7f1e6019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 -- 172.21.15.6:0/3529524062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e60073550 msgr2=0x7f1e6019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e60073550 0x7f1e6019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 -- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e6010c860 msgr2=0x7f1e601a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e6010c860 0x7f1e601a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 -- 172.21.15.6:0/3529524062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500095f0 con 0x7f1e60074b30 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.088+0000 7f1e5effd700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e970 tx=0x7f1e5400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.089+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54009790 con 0x7f1e60074b30 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.089+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e60112250 con 0x7f1e60074b30 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.089+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5400c7e0 con 0x7f1e60074b30 2023-11-17T00:06:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.089+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54003c70 con 0x7f1e60074b30 2023-11-17T00:06:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.089+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e601120c0 con 0x7f1e60074b30 2023-11-17T00:06:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.090+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e601023b0 con 0x7f1e60074b30 2023-11-17T00:06:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.092+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1e5400cc50 con 0x7f1e60074b30 2023-11-17T00:06:39.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.093+0000 7f1e5cff9700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 0x7f1e480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.093+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1e54014070 con 0x7f1e60074b30 2023-11-17T00:06:39.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.094+0000 7f1e5f7fe700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 0x7f1e480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.094+0000 7f1e5f7fe700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 0x7f1e480a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000b2e0 tx=0x7f1e50005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.097+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1e54096210 con 0x7f1e60074b30 2023-11-17T00:06:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:38 smithi087 ceph-mon[131182]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 341 B/s wr, 0 op/s 2023-11-17T00:06:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:38 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-17T00:06:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:38 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:39.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:38 smithi186 ceph-mon[131487]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 341 B/s wr, 0 op/s 2023-11-17T00:06:39.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:38 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-17T00:06:39.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:38 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-mon[97840]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 341 B/s wr, 0 op/s 2023-11-17T00:06:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-17T00:06:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:38 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:06:39.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.339+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e60061c10 con 0x7f1e480a1150 2023-11-17T00:06:39.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.366+0000 7f1e5cff9700 1 -- 172.21.15.6:0/3529524062 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e60061c10 con 0x7f1e480a1150 2023-11-17T00:06:39.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 msgr2=0x7f1e480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 0x7f1e480a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000b2e0 tx=0x7f1e50005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 msgr2=0x7f1e6019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e970 tx=0x7f1e5400ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 shutdown_connections 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1e480a1150 0x7f1e480a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e6010c860 0x7f1e601a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e60073550 0x7f1e6019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 --2- 172.21.15.6:0/3529524062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e60074b30 0x7f1e6019cb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.368+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 >> 172.21.15.6:0/3529524062 conn(0x7f1e600fb870 msgr2=0x7f1e600fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.369+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 shutdown_connections 2023-11-17T00:06:39.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.369+0000 7f1e65fff700 1 -- 172.21.15.6:0/3529524062 wait complete. 2023-11-17T00:06:39.388 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 -- 172.21.15.6:0/3049502708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110100290 msgr2=0x7f91101006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 --2- 172.21.15.6:0/3049502708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110100290 0x7f91101006a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9100009990 tx=0x7f9100009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 -- 172.21.15.6:0/3049502708 shutdown_connections 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 --2- 172.21.15.6:0/3049502708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110100290 0x7f91101006a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 --2- 172.21.15.6:0/3049502708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110101e10 0x7f91101086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 --2- 172.21.15.6:0/3049502708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101480 0x7f91101018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 -- 172.21.15.6:0/3049502708 >> 172.21.15.6:0/3049502708 conn(0x7f91100fb870 msgr2=0x7f91100fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:39.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 -- 172.21.15.6:0/3049502708 shutdown_connections 2023-11-17T00:06:39.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.595+0000 7f91165ff700 1 -- 172.21.15.6:0/3049502708 wait complete. 2023-11-17T00:06:39.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.596+0000 7f91165ff700 1 Processor -- start 2023-11-17T00:06:39.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.596+0000 7f91165ff700 1 -- start start 2023-11-17T00:06:39.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110100290 0x7f9110074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101e10 0x7f9110073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110073bc0 con 0x7f9110101480 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110073d30 con 0x7f9110100290 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f91165ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91101a00a0 con 0x7f9110101e10 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56254/0 (socket says 172.21.15.6:56254) 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 -- 172.21.15.6:0/1983693093 learned_addr learned my addr 172.21.15.6:0/1983693093 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910ffff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110100290 0x7f9110074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f9114b9c700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101e10 0x7f9110073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 -- 172.21.15.6:0/1983693093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101e10 msgr2=0x7f9110073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101e10 0x7f9110073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 -- 172.21.15.6:0/1983693093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110100290 msgr2=0x7f9110074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110100290 0x7f9110074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.597+0000 7f910f7fe700 1 -- 172.21.15.6:0/1983693093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000095f0 con 0x7f9110101480 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f910f7fe700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9a0 tx=0x7f910400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104009790 con 0x7f9110101480 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f910400cb70 con 0x7f9110101480 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911010bc40 con 0x7f9110101480 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104010ea0 con 0x7f9110101480 2023-11-17T00:06:39.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.598+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911010bab0 con 0x7f9110101480 2023-11-17T00:06:39.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.599+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110107950 con 0x7f9110101480 2023-11-17T00:06:39.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.601+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f910400cce0 con 0x7f9110101480 2023-11-17T00:06:39.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.602+0000 7f910d7fa700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 0x7f90f80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:39.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.603+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9104014070 con 0x7f9110101480 2023-11-17T00:06:39.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.603+0000 7f910ffff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 0x7f90f80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:39.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.603+0000 7f910ffff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 0x7f90f80a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f910000b2e0 tx=0x7f9100005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:39.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.606+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9104051730 con 0x7f9110101480 2023-11-17T00:06:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.862+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f91040965f0 con 0x7f9110101480 2023-11-17T00:06:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.863+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9110061c10 con 0x7f90f80a11d0 2023-11-17T00:06:39.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.867+0000 7f910d7fa700 1 -- 172.21.15.6:0/1983693093 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f9110061c10 con 0x7f90f80a11d0 2023-11-17T00:06:39.869 INFO:teuthology.orchestra.run.smithi006.stdout:/volumes/_nogroup/sv_1/ad45d892-14a7-476f-b7ff-60e41d566f52 2023-11-17T00:06:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 msgr2=0x7f90f80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 0x7f90f80a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f910000b2e0 tx=0x7f9100005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 msgr2=0x7f9110072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9a0 tx=0x7f910400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 shutdown_connections 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f90f80a11d0 0x7f90f80a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9110101e10 0x7f9110073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9110100290 0x7f9110074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 --2- 172.21.15.6:0/1983693093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9110101480 0x7f9110072ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 >> 172.21.15.6:0/1983693093 conn(0x7f91100fb870 msgr2=0x7f9110104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 shutdown_connections 2023-11-17T00:06:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:39.870+0000 7f91165ff700 1 -- 172.21.15.6:0/1983693093 wait complete. 2023-11-17T00:06:39.890 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:06:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 -- 172.21.15.6:0/3756762641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01002d0 msgr2=0x7f25f01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3756762641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01002d0 0x7f25f01006e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009990 tx=0x7f25e0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 -- 172.21.15.6:0/3756762641 shutdown_connections 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3756762641 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01002d0 0x7f25f01006e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3756762641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f0101e50 0x7f25f0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3756762641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f01014c0 0x7f25f0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 -- 172.21.15.6:0/3756762641 >> 172.21.15.6:0/3756762641 conn(0x7f25f00fb8b0 msgr2=0x7f25f00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 -- 172.21.15.6:0/3756762641 shutdown_connections 2023-11-17T00:06:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.094+0000 7f25f67ff700 1 -- 172.21.15.6:0/3756762641 wait complete. 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 Processor -- start 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 -- start start 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 0x7f25f00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f0101e50 0x7f25f0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0073bc0 con 0x7f25f01014c0 2023-11-17T00:06:40.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0073d30 con 0x7f25f01002d0 2023-11-17T00:06:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.095+0000 7f25f67ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f01a00e0 con 0x7f25f0101e50 2023-11-17T00:06:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 0x7f25f00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56274/0 (socket says 172.21.15.6:56274) 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 -- 172.21.15.6:0/3593564324 learned_addr learned my addr 172.21.15.6:0/3593564324 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 0x7f25f00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38114/0 (socket says 172.21.15.6:38114) 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25f4d9c700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f0101e50 0x7f25f0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 -- 172.21.15.6:0/3593564324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f0101e50 msgr2=0x7f25f0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f0101e50 0x7f25f0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 -- 172.21.15.6:0/3593564324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 msgr2=0x7f25f00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 0x7f25f00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 -- 172.21.15.6:0/3593564324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00095f0 con 0x7f25f01014c0 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.096+0000 7f25ef7fe700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d980 tx=0x7f25e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.097+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004220 con 0x7f25f01014c0 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.097+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4007640 con 0x7f25f01014c0 2023-11-17T00:06:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.097+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f010bca0 con 0x7f25f01014c0 2023-11-17T00:06:40.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.097+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4005690 con 0x7f25f01014c0 2023-11-17T00:06:40.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.097+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f010bb10 con 0x7f25f01014c0 2023-11-17T00:06:40.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.098+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f0107800 con 0x7f25f01014c0 2023-11-17T00:06:40.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.100+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f25e4004750 con 0x7f25f01014c0 2023-11-17T00:06:40.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.101+0000 7f25ed7fa700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 0x7f25d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.101+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f25e40cca90 con 0x7f25f01014c0 2023-11-17T00:06:40.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.101+0000 7f25effff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 0x7f25d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.102+0000 7f25effff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 0x7f25d80a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25e000b2e0 tx=0x7f25e0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:40.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.105+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f25e4095e30 con 0x7f25f01014c0 2023-11-17T00:06:40.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:39 smithi186 ceph-mon[131487]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:39 smithi186 ceph-mon[131487]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:39 smithi006 ceph-mon[97840]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:39 smithi006 ceph-mon[97840]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:39 smithi087 ceph-mon[131182]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.380 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:39 smithi087 ceph-mon[131182]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.365+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25f002cfe0 con 0x7f25f01014c0 2023-11-17T00:06:40.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.366+0000 7f25ed7fa700 1 -- 172.21.15.6:0/3593564324 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f25e4016020 con 0x7f25f01014c0 2023-11-17T00:06:40.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:40.381 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 msgr2=0x7f25d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 0x7f25d80a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25e000b2e0 tx=0x7f25e0005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 msgr2=0x7f25f0072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d980 tx=0x7f25e400dc90 comp rx=0 tx=0).stop 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 shutdown_connections 2023-11-17T00:06:40.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f25d80a1260 0x7f25d80a3710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f0101e50 0x7f25f0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25f01002d0 0x7f25f00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 --2- 172.21.15.6:0/3593564324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25f01014c0 0x7f25f0072ee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 >> 172.21.15.6:0/3593564324 conn(0x7f25f00fb8b0 msgr2=0x7f25f0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 shutdown_connections 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.368+0000 7f25f67ff700 1 -- 172.21.15.6:0/3593564324 wait complete. 2023-11-17T00:06:40.383 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:06:40.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:06:40.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 -- 172.21.15.6:0/2722952736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101340 msgr2=0x7fedf81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 --2- 172.21.15.6:0/2722952736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101340 0x7fedf81017b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fedec009990 tx=0x7fedec009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 -- 172.21.15.6:0/2722952736 shutdown_connections 2023-11-17T00:06:40.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 --2- 172.21.15.6:0/2722952736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101cf0 0x7fedf8108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 --2- 172.21.15.6:0/2722952736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 0x7fedf81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 --2- 172.21.15.6:0/2722952736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101340 0x7fedf81017b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 -- 172.21.15.6:0/2722952736 >> 172.21.15.6:0/2722952736 conn(0x7fedf80fb6b0 msgr2=0x7fedf80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 -- 172.21.15.6:0/2722952736 shutdown_connections 2023-11-17T00:06:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.593+0000 7fedf759e700 1 -- 172.21.15.6:0/2722952736 wait complete. 2023-11-17T00:06:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.594+0000 7fedf759e700 1 Processor -- start 2023-11-17T00:06:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 -- start start 2023-11-17T00:06:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 0x7fedf8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 0x7fedf8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819fb70 con 0x7fedf8101cf0 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819fcb0 con 0x7fedf8101340 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf759e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819fe40 con 0x7fedf81000b0 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56278/0 (socket says 172.21.15.6:56278) 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf6d9d700 1 -- 172.21.15.6:0/2400650488 learned_addr learned my addr 172.21.15.6:0/2400650488 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf5d9b700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 0x7fedf8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.595+0000 7fedf659c700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 0x7fedf8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 -- 172.21.15.6:0/2400650488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 msgr2=0x7fedf8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 0x7fedf8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 -- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 msgr2=0x7fedf819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 -- 172.21.15.6:0/2400650488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec0095f0 con 0x7fedf8101340 2023-11-17T00:06:40.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf6d9d700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:40.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf5d9b700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 0x7fedf8198530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b680 tx=0x7fedec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:40.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec01d070 con 0x7fedf8101340 2023-11-17T00:06:40.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf810bb00 con 0x7fedf8101340 2023-11-17T00:06:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedec021410 con 0x7fedf8101340 2023-11-17T00:06:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec017650 con 0x7fedf8101340 2023-11-17T00:06:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.596+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf810b970 con 0x7fedf8101340 2023-11-17T00:06:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.597+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf8067970 con 0x7fedf8101340 2023-11-17T00:06:40.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.599+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fedec021810 con 0x7fedf8101340 2023-11-17T00:06:40.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.602+0000 7fede77fe700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 0x7fede00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:40.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.602+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fedec0ce680 con 0x7fedf8101340 2023-11-17T00:06:40.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.602+0000 7fedf659c700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 0x7fede00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:40.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.602+0000 7fedf659c700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 0x7fede00a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fede8009c60 tx=0x7fede8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.603+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fedec097ab0 con 0x7fedf8101340 2023-11-17T00:06:40.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.813+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedf81a0a10 con 0x7fedf8101340 2023-11-17T00:06:40.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.816+0000 7fede77fe700 1 -- 172.21.15.6:0/2400650488 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fedec09b0d0 con 0x7fedf8101340 2023-11-17T00:06:40.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:40.818 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:40.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 msgr2=0x7fede00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 0x7fede00a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fede8009c60 tx=0x7fede8009260 comp rx=0 tx=0).stop 2023-11-17T00:06:40.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 msgr2=0x7fedf8198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 0x7fedf8198530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b680 tx=0x7fedec009580 comp rx=0 tx=0).stop 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 shutdown_connections 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fede00a11e0 0x7fede00a3690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedf8101cf0 0x7fedf819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedf81000b0 0x7fedf8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 --2- 172.21.15.6:0/2400650488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf8101340 0x7fedf8198530 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.817+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 >> 172.21.15.6:0/2400650488 conn(0x7fedf80fb6b0 msgr2=0x7fedf80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.818+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 shutdown_connections 2023-11-17T00:06:40.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:40.818+0000 7fedf759e700 1 -- 172.21.15.6:0/2400650488 wait complete. 2023-11-17T00:06:40.839 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-17T00:06:40.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-17T00:06:41.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.043+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2048734404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 msgr2=0x7fcb481006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.043+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2048734404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb481006e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009990 tx=0x7fcb38009ca0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2048734404 shutdown_connections 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2048734404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb481006e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2048734404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 0x7fcb48108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2048734404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 0x7fcb48101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2048734404 >> 172.21.15.6:0/2048734404 conn(0x7fcb480fb8b0 msgr2=0x7fcb480fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2048734404 shutdown_connections 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.044+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2048734404 wait complete. 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 Processor -- start 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 -- start start 2023-11-17T00:06:41.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 0x7fcb48072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 0x7fcb48073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48073bc0 con 0x7fcb481002d0 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48073d30 con 0x7fcb481014c0 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb481a0100 con 0x7fcb48101e50 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56306/0 (socket says 172.21.15.6:56306) 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 -- 172.21.15.6:0/2394038029 learned_addr learned my addr 172.21.15.6:0/2394038029 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb46d9d700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 0x7fcb48073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 0x7fcb48072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 -- 172.21.15.6:0/2394038029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 msgr2=0x7fcb48073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 0x7fcb48073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 -- 172.21.15.6:0/2394038029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 msgr2=0x7fcb48072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 0x7fcb48072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 -- 172.21.15.6:0/2394038029 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380095f0 con 0x7fcb481002d0 2023-11-17T00:06:41.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb4659c700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb38005740 tx=0x7fcb38005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:41.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.046+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3801d070 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.047+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4810bca0 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.047+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb38021410 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.047+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38017650 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.047+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4810bb10 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.048+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb48107860 con 0x7fcb481002d0 2023-11-17T00:06:41.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.050+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcb38021580 con 0x7fcb481002d0 2023-11-17T00:06:41.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.051+0000 7fcb3f7fe700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 0x7fcb340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.051+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcb380cdde0 con 0x7fcb481002d0 2023-11-17T00:06:41.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.051+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 0x7fcb340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.052+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 0x7fcb340a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009c60 tx=0x7fcb30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:41.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.055+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb38097210 con 0x7fcb481002d0 2023-11-17T00:06:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: pgmap v307: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 1 op/s 2023-11-17T00:06:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-17T00:06:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3593564324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:40 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2400650488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: pgmap v307: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 1 op/s 2023-11-17T00:06:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-17T00:06:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3593564324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:40 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2400650488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.267+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb48060f30 con 0x7fcb340a11e0 2023-11-17T00:06:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.283+0000 7fcb3f7fe700 1 -- 172.21.15.6:0/2394038029 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421919 (secure 0 0 0) 0x7fcb48060f30 con 0x7fcb340a11e0 2023-11-17T00:06:41.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:41.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 msgr2=0x7fcb340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 0x7fcb340a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009c60 tx=0x7fcb30009260 comp rx=0 tx=0).stop 2023-11-17T00:06:41.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 msgr2=0x7fcb480748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb38005740 tx=0x7fcb38005770 comp rx=0 tx=0).stop 2023-11-17T00:06:41.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 shutdown_connections 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fcb340a11e0 0x7fcb340a3690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb481002d0 0x7fcb480748a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48101e50 0x7fcb48073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 --2- 172.21.15.6:0/2394038029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb481014c0 0x7fcb48072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 >> 172.21.15.6:0/2394038029 conn(0x7fcb480fb8b0 msgr2=0x7fcb481049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 shutdown_connections 2023-11-17T00:06:41.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.288+0000 7fcb4cbff700 1 -- 172.21.15.6:0/2394038029 wait complete. 2023-11-17T00:06:41.298 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:06:41.309 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-11-17T00:06:39.744599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612341,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":147,"num_read_kb":131,"num_write":236,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79584,"kb_used_data":11392,"kb_used_omap":0,"kb_used_meta":68096,"kb_avail":1124862240,"statfs":{"total":1151940427776,"available":1151858933760,"internally_reserved":0,"allocated":11665408,"data_stored":7457287,"data_compressed":247614,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69730304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19611,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":10,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606911+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:06:17.606911+0000","last_peered":"2023-11-17T00:06:17.606911+0000","last_clean":"2023-11-17T00:06:17.606911+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:06:17.606911+0000","last_undegraded":"2023-11-17T00:06:17.606911+0000","last_fullsized":"2023-11-17T00:06:17.606911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:59:51.597677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606065+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:06:17.606065+0000","last_peered":"2023-11-17T00:06:17.606065+0000","last_clean":"2023-11-17T00:06:17.606065+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:06:17.606065+0000","last_undegraded":"2023-11-17T00:06:17.606065+0000","last_fullsized":"2023-11-17T00:06:17.606065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:16:04.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605953+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:06:17.605953+0000","last_peered":"2023-11-17T00:06:17.605953+0000","last_clean":"2023-11-17T00:06:17.605953+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:06:17.605953+0000","last_undegraded":"2023-11-17T00:06:17.605953+0000","last_fullsized":"2023-11-17T00:06:17.605953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:34:09.527417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215164+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:06:18.215164+0000","last_peered":"2023-11-17T00:06:18.215164+0000","last_clean":"2023-11-17T00:06:18.215164+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:06:18.215164+0000","last_undegraded":"2023-11-17T00:06:18.215164+0000","last_fullsized":"2023-11-17T00:06:18.215164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:25.724124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214573+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:06:18.214573+0000","last_peered":"2023-11-17T00:06:18.214573+0000","last_clean":"2023-11-17T00:06:18.214573+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:06:18.214573+0000","last_undegraded":"2023-11-17T00:06:18.214573+0000","last_fullsized":"2023-11-17T00:06:18.214573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:59:56.856981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214611+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:06:18.214611+0000","last_peered":"2023-11-17T00:06:18.214611+0000","last_clean":"2023-11-17T00:06:18.214611+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:06:18.214611+0000","last_undegraded":"2023-11-17T00:06:18.214611+0000","last_fullsized":"2023-11-17T00:06:18.214611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:09:42.846932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606243+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:06:17.606243+0000","last_peered":"2023-11-17T00:06:17.606243+0000","last_clean":"2023-11-17T00:06:17.606243+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:06:17.606243+0000","last_undegraded":"2023-11-17T00:06:17.606243+0000","last_fullsized":"2023-11-17T00:06:17.606243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:39:59.072403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280390+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:06:18.280390+0000","last_peered":"2023-11-17T00:06:18.280390+0000","last_clean":"2023-11-17T00:06:18.280390+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:06:18.280390+0000","last_undegraded":"2023-11-17T00:06:18.280390+0000","last_fullsized":"2023-11-17T00:06:18.280390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:17:15.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216147+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:06:18.216147+0000","last_peered":"2023-11-17T00:06:18.216147+0000","last_clean":"2023-11-17T00:06:18.216147+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:06:18.216147+0000","last_undegraded":"2023-11-17T00:06:18.216147+0000","last_fullsized":"2023-11-17T00:06:18.216147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:06.166420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215489+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:06:18.215489+0000","last_peered":"2023-11-17T00:06:18.215489+0000","last_clean":"2023-11-17T00:06:18.215489+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:06:18.215489+0000","last_undegraded":"2023-11-17T00:06:18.215489+0000","last_fullsized":"2023-11-17T00:06:18.215489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:55:34.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606830+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:06:17.606830+0000","last_peered":"2023-11-17T00:06:17.606830+0000","last_clean":"2023-11-17T00:06:17.606830+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:06:17.606830+0000","last_undegraded":"2023-11-17T00:06:17.606830+0000","last_fullsized":"2023-11-17T00:06:17.606830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:48:01.654901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215022+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:06:18.215022+0000","last_peered":"2023-11-17T00:06:18.215022+0000","last_clean":"2023-11-17T00:06:18.215022+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:06:18.215022+0000","last_undegraded":"2023-11-17T00:06:18.215022+0000","last_fullsized":"2023-11-17T00:06:18.215022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:15.312936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214430+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:06:18.214430+0000","last_peered":"2023-11-17T00:06:18.214430+0000","last_clean":"2023-11-17T00:06:18.214430+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:06:18.214430+0000","last_undegraded":"2023-11-17T00:06:18.214430+0000","last_fullsized":"2023-11-17T00:06:18.214430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:31:08.188039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605613+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:06:17.605613+0000","last_peered":"2023-11-17T00:06:17.605613+0000","last_clean":"2023-11-17T00:06:17.605613+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:06:17.605613+0000","last_undegraded":"2023-11-17T00:06:17.605613+0000","last_fullsized":"2023-11-17T00:06:17.605613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:08:28.258146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605775+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:06:17.605775+0000","last_peered":"2023-11-17T00:06:17.605775+0000","last_clean":"2023-11-17T00:06:17.605775+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:06:17.605775+0000","last_undegraded":"2023-11-17T00:06:17.605775+0000","last_fullsized":"2023-11-17T00:06:17.605775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:44:47.871198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216141+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:06:18.216141+0000","last_peered":"2023-11-17T00:06:18.216141+0000","last_clean":"2023-11-17T00:06:18.216141+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:06:18.216141+0000","last_undegraded":"2023-11-17T00:06:18.216141+0000","last_fullsized":"2023-11-17T00:06:18.216141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:56:05.011055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606750+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:06:17.606750+0000","last_peered":"2023-11-17T00:06:17.606750+0000","last_clean":"2023-11-17T00:06:17.606750+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:06:17.606750+0000","last_undegraded":"2023-11-17T00:06:17.606750+0000","last_fullsized":"2023-11-17T00:06:17.606750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:49:56.168140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014883+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:06:18.014883+0000","last_peered":"2023-11-17T00:06:18.014883+0000","last_clean":"2023-11-17T00:06:18.014883+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:06:18.014883+0000","last_undegraded":"2023-11-17T00:06:18.014883+0000","last_fullsized":"2023-11-17T00:06:18.014883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:04:11.558089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216547+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:06:18.216547+0000","last_peered":"2023-11-17T00:06:18.216547+0000","last_clean":"2023-11-17T00:06:18.216547+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:06:18.216547+0000","last_undegraded":"2023-11-17T00:06:18.216547+0000","last_fullsized":"2023-11-17T00:06:18.216547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:19:03.058159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216065+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:06:18.216065+0000","last_peered":"2023-11-17T00:06:18.216065+0000","last_clean":"2023-11-17T00:06:18.216065+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:06:18.216065+0000","last_undegraded":"2023-11-17T00:06:18.216065+0000","last_fullsized":"2023-11-17T00:06:18.216065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:46.436458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.015002+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:06:18.015002+0000","last_peered":"2023-11-17T00:06:18.015002+0000","last_clean":"2023-11-17T00:06:18.015002+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:06:18.015002+0000","last_undegraded":"2023-11-17T00:06:18.015002+0000","last_fullsized":"2023-11-17T00:06:18.015002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:46:03.054290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280846+0000","last_change":"2023-11-17T00:06:12.628519+0000","last_active":"2023-11-17T00:06:18.280846+0000","last_peered":"2023-11-17T00:06:18.280846+0000","last_clean":"2023-11-17T00:06:18.280846+0000","last_became_active":"2023-11-17T00:06:12.627360+0000","last_became_peered":"2023-11-17T00:06:12.627360+0000","last_unstale":"2023-11-17T00:06:18.280846+0000","last_undegraded":"2023-11-17T00:06:18.280846+0000","last_fullsized":"2023-11-17T00:06:18.280846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:45:56.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214319+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:06:18.214319+0000","last_peered":"2023-11-17T00:06:18.214319+0000","last_clean":"2023-11-17T00:06:18.214319+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:06:18.214319+0000","last_undegraded":"2023-11-17T00:06:18.214319+0000","last_fullsized":"2023-11-17T00:06:18.214319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:27:48.580204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281003+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:06:18.281003+0000","last_peered":"2023-11-17T00:06:18.281003+0000","last_clean":"2023-11-17T00:06:18.281003+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:06:18.281003+0000","last_undegraded":"2023-11-17T00:06:18.281003+0000","last_fullsized":"2023-11-17T00:06:18.281003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:03:47.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606195+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:06:17.606195+0000","last_peered":"2023-11-17T00:06:17.606195+0000","last_clean":"2023-11-17T00:06:17.606195+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:06:17.606195+0000","last_undegraded":"2023-11-17T00:06:17.606195+0000","last_fullsized":"2023-11-17T00:06:17.606195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:28:19.627128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214931+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:06:18.214931+0000","last_peered":"2023-11-17T00:06:18.214931+0000","last_clean":"2023-11-17T00:06:18.214931+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:06:18.214931+0000","last_undegraded":"2023-11-17T00:06:18.214931+0000","last_fullsized":"2023-11-17T00:06:18.214931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:48:29.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215799+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:06:18.215799+0000","last_peered":"2023-11-17T00:06:18.215799+0000","last_clean":"2023-11-17T00:06:18.215799+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:06:18.215799+0000","last_undegraded":"2023-11-17T00:06:18.215799+0000","last_fullsized":"2023-11-17T00:06:18.215799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:35.530902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215028+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:06:18.215028+0000","last_peered":"2023-11-17T00:06:18.215028+0000","last_clean":"2023-11-17T00:06:18.215028+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:06:18.215028+0000","last_undegraded":"2023-11-17T00:06:18.215028+0000","last_fullsized":"2023-11-17T00:06:18.215028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:56.991464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214632+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:06:18.214632+0000","last_peered":"2023-11-17T00:06:18.214632+0000","last_clean":"2023-11-17T00:06:18.214632+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:06:18.214632+0000","last_undegraded":"2023-11-17T00:06:18.214632+0000","last_fullsized":"2023-11-17T00:06:18.214632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:33:01.556713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215066+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:06:18.215066+0000","last_peered":"2023-11-17T00:06:18.215066+0000","last_clean":"2023-11-17T00:06:18.215066+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:06:18.215066+0000","last_undegraded":"2023-11-17T00:06:18.215066+0000","last_fullsized":"2023-11-17T00:06:18.215066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:25:59.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216253+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:06:18.216253+0000","last_peered":"2023-11-17T00:06:18.216253+0000","last_clean":"2023-11-17T00:06:18.216253+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:06:18.216253+0000","last_undegraded":"2023-11-17T00:06:18.216253+0000","last_fullsized":"2023-11-17T00:06:18.216253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:57:51.933886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216310+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:06:18.216310+0000","last_peered":"2023-11-17T00:06:18.216310+0000","last_clean":"2023-11-17T00:06:18.216310+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:06:18.216310+0000","last_undegraded":"2023-11-17T00:06:18.216310+0000","last_fullsized":"2023-11-17T00:06:18.216310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:01.254910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215196+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:06:18.215196+0000","last_peered":"2023-11-17T00:06:18.215196+0000","last_clean":"2023-11-17T00:06:18.215196+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:06:18.215196+0000","last_undegraded":"2023-11-17T00:06:18.215196+0000","last_fullsized":"2023-11-17T00:06:18.215196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:39:02.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215163+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:06:18.215163+0000","last_peered":"2023-11-17T00:06:18.215163+0000","last_clean":"2023-11-17T00:06:18.215163+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:06:18.215163+0000","last_undegraded":"2023-11-17T00:06:18.215163+0000","last_fullsized":"2023-11-17T00:06:18.215163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:40:11.905147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216186+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:06:18.216186+0000","last_peered":"2023-11-17T00:06:18.216186+0000","last_clean":"2023-11-17T00:06:18.216186+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:06:18.216186+0000","last_undegraded":"2023-11-17T00:06:18.216186+0000","last_fullsized":"2023-11-17T00:06:18.216186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:45:47.937434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214558+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:06:18.214558+0000","last_peered":"2023-11-17T00:06:18.214558+0000","last_clean":"2023-11-17T00:06:18.214558+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:06:18.214558+0000","last_undegraded":"2023-11-17T00:06:18.214558+0000","last_fullsized":"2023-11-17T00:06:18.214558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:18:01.928364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215417+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:06:18.215417+0000","last_peered":"2023-11-17T00:06:18.215417+0000","last_clean":"2023-11-17T00:06:18.215417+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:06:18.215417+0000","last_undegraded":"2023-11-17T00:06:18.215417+0000","last_fullsized":"2023-11-17T00:06:18.215417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:28:49.567187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606751+0000","last_change":"2023-11-17T00:06:17.102667+0000","last_active":"2023-11-17T00:06:17.606751+0000","last_peered":"2023-11-17T00:06:17.606751+0000","last_clean":"2023-11-17T00:06:17.606751+0000","last_became_active":"2023-11-17T00:06:17.102273+0000","last_became_peered":"2023-11-17T00:06:17.102273+0000","last_unstale":"2023-11-17T00:06:17.606751+0000","last_undegraded":"2023-11-17T00:06:17.606751+0000","last_fullsized":"2023-11-17T00:06:17.606751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:06:18.674176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214070+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:06:18.214070+0000","last_peered":"2023-11-17T00:06:18.214070+0000","last_clean":"2023-11-17T00:06:18.214070+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:06:18.214070+0000","last_undegraded":"2023-11-17T00:06:18.214070+0000","last_fullsized":"2023-11-17T00:06:18.214070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:08:51.997009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280220+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:06:18.280220+0000","last_peered":"2023-11-17T00:06:18.280220+0000","last_clean":"2023-11-17T00:06:18.280220+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:06:18.280220+0000","last_undegraded":"2023-11-17T00:06:18.280220+0000","last_fullsized":"2023-11-17T00:06:18.280220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:18:24.283794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606112+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:06:17.606112+0000","last_peered":"2023-11-17T00:06:17.606112+0000","last_clean":"2023-11-17T00:06:17.606112+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:06:17.606112+0000","last_undegraded":"2023-11-17T00:06:17.606112+0000","last_fullsized":"2023-11-17T00:06:17.606112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:30:08.247199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214924+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:06:18.214924+0000","last_peered":"2023-11-17T00:06:18.214924+0000","last_clean":"2023-11-17T00:06:18.214924+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:06:18.214924+0000","last_undegraded":"2023-11-17T00:06:18.214924+0000","last_fullsized":"2023-11-17T00:06:18.214924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:30:02.850350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014324+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:06:18.014324+0000","last_peered":"2023-11-17T00:06:18.014324+0000","last_clean":"2023-11-17T00:06:18.014324+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:06:18.014324+0000","last_undegraded":"2023-11-17T00:06:18.014324+0000","last_fullsized":"2023-11-17T00:06:18.014324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:13:23.416055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280538+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:06:18.280538+0000","last_peered":"2023-11-17T00:06:18.280538+0000","last_clean":"2023-11-17T00:06:18.280538+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:06:18.280538+0000","last_undegraded":"2023-11-17T00:06:18.280538+0000","last_fullsized":"2023-11-17T00:06:18.280538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:38:35.251669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215393+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:06:18.215393+0000","last_peered":"2023-11-17T00:06:18.215393+0000","last_clean":"2023-11-17T00:06:18.215393+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:06:18.215393+0000","last_undegraded":"2023-11-17T00:06:18.215393+0000","last_fullsized":"2023-11-17T00:06:18.215393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:36:09.826273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216005+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:06:18.216005+0000","last_peered":"2023-11-17T00:06:18.216005+0000","last_clean":"2023-11-17T00:06:18.216005+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:06:18.216005+0000","last_undegraded":"2023-11-17T00:06:18.216005+0000","last_fullsized":"2023-11-17T00:06:18.216005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:29:15.971648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014232+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:06:18.014232+0000","last_peered":"2023-11-17T00:06:18.014232+0000","last_clean":"2023-11-17T00:06:18.014232+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:06:18.014232+0000","last_undegraded":"2023-11-17T00:06:18.014232+0000","last_fullsized":"2023-11-17T00:06:18.014232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:44:46.704510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606766+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:06:17.606766+0000","last_peered":"2023-11-17T00:06:17.606766+0000","last_clean":"2023-11-17T00:06:17.606766+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:06:17.606766+0000","last_undegraded":"2023-11-17T00:06:17.606766+0000","last_fullsized":"2023-11-17T00:06:17.606766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:30:05.357211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606209+0000","last_change":"2023-11-17T00:06:12.638172+0000","last_active":"2023-11-17T00:06:17.606209+0000","last_peered":"2023-11-17T00:06:17.606209+0000","last_clean":"2023-11-17T00:06:17.606209+0000","last_became_active":"2023-11-17T00:06:12.637227+0000","last_became_peered":"2023-11-17T00:06:12.637227+0000","last_unstale":"2023-11-17T00:06:17.606209+0000","last_undegraded":"2023-11-17T00:06:17.606209+0000","last_fullsized":"2023-11-17T00:06:17.606209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:44:00.879902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215124+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:06:18.215124+0000","last_peered":"2023-11-17T00:06:18.215124+0000","last_clean":"2023-11-17T00:06:18.215124+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:06:18.215124+0000","last_undegraded":"2023-11-17T00:06:18.215124+0000","last_fullsized":"2023-11-17T00:06:18.215124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:50.709860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215696+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:06:18.215696+0000","last_peered":"2023-11-17T00:06:18.215696+0000","last_clean":"2023-11-17T00:06:18.215696+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:06:18.215696+0000","last_undegraded":"2023-11-17T00:06:18.215696+0000","last_fullsized":"2023-11-17T00:06:18.215696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:56:21.913956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215253+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:06:18.215253+0000","last_peered":"2023-11-17T00:06:18.215253+0000","last_clean":"2023-11-17T00:06:18.215253+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:06:18.215253+0000","last_undegraded":"2023-11-17T00:06:18.215253+0000","last_fullsized":"2023-11-17T00:06:18.215253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:34:55.155458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014343+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:06:18.014343+0000","last_peered":"2023-11-17T00:06:18.014343+0000","last_clean":"2023-11-17T00:06:18.014343+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:06:18.014343+0000","last_undegraded":"2023-11-17T00:06:18.014343+0000","last_fullsized":"2023-11-17T00:06:18.014343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:51.824064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'16","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:38.361186+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:06:38.361186+0000","last_peered":"2023-11-17T00:06:38.361186+0000","last_clean":"2023-11-17T00:06:38.361186+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:06:38.361186+0000","last_undegraded":"2023-11-17T00:06:38.361186+0000","last_fullsized":"2023-11-17T00:06:38.361186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:28:44.761111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21175,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":16,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215470+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:06:18.215470+0000","last_peered":"2023-11-17T00:06:18.215470+0000","last_clean":"2023-11-17T00:06:18.215470+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:06:18.215470+0000","last_undegraded":"2023-11-17T00:06:18.215470+0000","last_fullsized":"2023-11-17T00:06:18.215470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:16.899631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216226+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:06:18.216226+0000","last_peered":"2023-11-17T00:06:18.216226+0000","last_clean":"2023-11-17T00:06:18.216226+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:06:18.216226+0000","last_undegraded":"2023-11-17T00:06:18.216226+0000","last_fullsized":"2023-11-17T00:06:18.216226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:16:56.547530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216332+0000","last_change":"2023-11-17T00:06:12.107382+0000","last_active":"2023-11-17T00:06:18.216332+0000","last_peered":"2023-11-17T00:06:18.216332+0000","last_clean":"2023-11-17T00:06:18.216332+0000","last_became_active":"2023-11-17T00:06:12.105773+0000","last_became_peered":"2023-11-17T00:06:12.105773+0000","last_unstale":"2023-11-17T00:06:18.216332+0000","last_undegraded":"2023-11-17T00:06:18.216332+0000","last_fullsized":"2023-11-17T00:06:18.216332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:18.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214106+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:06:18.214106+0000","last_peered":"2023-11-17T00:06:18.214106+0000","last_clean":"2023-11-17T00:06:18.214106+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:06:18.214106+0000","last_undegraded":"2023-11-17T00:06:18.214106+0000","last_fullsized":"2023-11-17T00:06:18.214106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:09:28.039505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606311+0000","last_change":"2023-11-17T00:06:17.104407+0000","last_active":"2023-11-17T00:06:17.606311+0000","last_peered":"2023-11-17T00:06:17.606311+0000","last_clean":"2023-11-17T00:06:17.606311+0000","last_became_active":"2023-11-17T00:06:17.103089+0000","last_became_peered":"2023-11-17T00:06:17.103089+0000","last_unstale":"2023-11-17T00:06:17.606311+0000","last_undegraded":"2023-11-17T00:06:17.606311+0000","last_fullsized":"2023-11-17T00:06:17.606311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:45:58.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215544+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:06:18.215544+0000","last_peered":"2023-11-17T00:06:18.215544+0000","last_clean":"2023-11-17T00:06:18.215544+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:06:18.215544+0000","last_undegraded":"2023-11-17T00:06:18.215544+0000","last_fullsized":"2023-11-17T00:06:18.215544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:03:32.099745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606663+0000","last_change":"2023-11-17T00:06:12.632671+0000","last_active":"2023-11-17T00:06:17.606663+0000","last_peered":"2023-11-17T00:06:17.606663+0000","last_clean":"2023-11-17T00:06:17.606663+0000","last_became_active":"2023-11-17T00:06:12.632332+0000","last_became_peered":"2023-11-17T00:06:12.632332+0000","last_unstale":"2023-11-17T00:06:17.606663+0000","last_undegraded":"2023-11-17T00:06:17.606663+0000","last_fullsized":"2023-11-17T00:06:17.606663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:17.846720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014853+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:06:18.014853+0000","last_peered":"2023-11-17T00:06:18.014853+0000","last_clean":"2023-11-17T00:06:18.014853+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:06:18.014853+0000","last_undegraded":"2023-11-17T00:06:18.014853+0000","last_fullsized":"2023-11-17T00:06:18.014853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:22:02.039903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215789+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:06:18.215789+0000","last_peered":"2023-11-17T00:06:18.215789+0000","last_clean":"2023-11-17T00:06:18.215789+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:06:18.215789+0000","last_undegraded":"2023-11-17T00:06:18.215789+0000","last_fullsized":"2023-11-17T00:06:18.215789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:09:34.331979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606969+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:06:17.606969+0000","last_peered":"2023-11-17T00:06:17.606969+0000","last_clean":"2023-11-17T00:06:17.606969+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:06:17.606969+0000","last_undegraded":"2023-11-17T00:06:17.606969+0000","last_fullsized":"2023-11-17T00:06:17.606969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:54:48.901609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214492+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:06:18.214492+0000","last_peered":"2023-11-17T00:06:18.214492+0000","last_clean":"2023-11-17T00:06:18.214492+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:06:18.214492+0000","last_undegraded":"2023-11-17T00:06:18.214492+0000","last_fullsized":"2023-11-17T00:06:18.214492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:27:54.479757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214354+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:06:18.214354+0000","last_peered":"2023-11-17T00:06:18.214354+0000","last_clean":"2023-11-17T00:06:18.214354+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:06:18.214354+0000","last_undegraded":"2023-11-17T00:06:18.214354+0000","last_fullsized":"2023-11-17T00:06:18.214354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:51.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214640+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:06:18.214640+0000","last_peered":"2023-11-17T00:06:18.214640+0000","last_clean":"2023-11-17T00:06:18.214640+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:06:18.214640+0000","last_undegraded":"2023-11-17T00:06:18.214640+0000","last_fullsized":"2023-11-17T00:06:18.214640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:19:10.256370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214565+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:06:18.214565+0000","last_peered":"2023-11-17T00:06:18.214565+0000","last_clean":"2023-11-17T00:06:18.214565+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:06:18.214565+0000","last_undegraded":"2023-11-17T00:06:18.214565+0000","last_fullsized":"2023-11-17T00:06:18.214565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:23:36.703118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215049+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:06:18.215049+0000","last_peered":"2023-11-17T00:06:18.215049+0000","last_clean":"2023-11-17T00:06:18.215049+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:06:18.215049+0000","last_undegraded":"2023-11-17T00:06:18.215049+0000","last_fullsized":"2023-11-17T00:06:18.215049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:04:44.010870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606687+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:06:17.606687+0000","last_peered":"2023-11-17T00:06:17.606687+0000","last_clean":"2023-11-17T00:06:17.606687+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:06:17.606687+0000","last_undegraded":"2023-11-17T00:06:17.606687+0000","last_fullsized":"2023-11-17T00:06:17.606687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:03:50.330931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214973+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:06:18.214973+0000","last_peered":"2023-11-17T00:06:18.214973+0000","last_clean":"2023-11-17T00:06:18.214973+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:06:18.214973+0000","last_undegraded":"2023-11-17T00:06:18.214973+0000","last_fullsized":"2023-11-17T00:06:18.214973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:19:37.010947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014642+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:06:18.014642+0000","last_peered":"2023-11-17T00:06:18.014642+0000","last_clean":"2023-11-17T00:06:18.014642+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:06:18.014642+0000","last_undegraded":"2023-11-17T00:06:18.014642+0000","last_fullsized":"2023-11-17T00:06:18.014642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:01:46.650139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215360+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:06:18.215360+0000","last_peered":"2023-11-17T00:06:18.215360+0000","last_clean":"2023-11-17T00:06:18.215360+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:06:18.215360+0000","last_undegraded":"2023-11-17T00:06:18.215360+0000","last_fullsized":"2023-11-17T00:06:18.215360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:57:54.487416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214288+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:06:18.214288+0000","last_peered":"2023-11-17T00:06:18.214288+0000","last_clean":"2023-11-17T00:06:18.214288+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:06:18.214288+0000","last_undegraded":"2023-11-17T00:06:18.214288+0000","last_fullsized":"2023-11-17T00:06:18.214288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:03:22.517337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606446+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:06:17.606446+0000","last_peered":"2023-11-17T00:06:17.606446+0000","last_clean":"2023-11-17T00:06:17.606446+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:06:17.606446+0000","last_undegraded":"2023-11-17T00:06:17.606446+0000","last_fullsized":"2023-11-17T00:06:17.606446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:46:19.855956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215139+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:06:18.215139+0000","last_peered":"2023-11-17T00:06:18.215139+0000","last_clean":"2023-11-17T00:06:18.215139+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:06:18.215139+0000","last_undegraded":"2023-11-17T00:06:18.215139+0000","last_fullsized":"2023-11-17T00:06:18.215139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:37:43.731363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214302+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:06:18.214302+0000","last_peered":"2023-11-17T00:06:18.214302+0000","last_clean":"2023-11-17T00:06:18.214302+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:06:18.214302+0000","last_undegraded":"2023-11-17T00:06:18.214302+0000","last_fullsized":"2023-11-17T00:06:18.214302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:47:36.720175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215201+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:06:18.215201+0000","last_peered":"2023-11-17T00:06:18.215201+0000","last_clean":"2023-11-17T00:06:18.215201+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:06:18.215201+0000","last_undegraded":"2023-11-17T00:06:18.215201+0000","last_fullsized":"2023-11-17T00:06:18.215201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:47:33.653002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.213879+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:06:18.213879+0000","last_peered":"2023-11-17T00:06:18.213879+0000","last_clean":"2023-11-17T00:06:18.213879+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:06:18.213879+0000","last_undegraded":"2023-11-17T00:06:18.213879+0000","last_fullsized":"2023-11-17T00:06:18.213879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:23:27.835317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606830+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:06:17.606830+0000","last_peered":"2023-11-17T00:06:17.606830+0000","last_clean":"2023-11-17T00:06:17.606830+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:06:17.606830+0000","last_undegraded":"2023-11-17T00:06:17.606830+0000","last_fullsized":"2023-11-17T00:06:17.606830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:05:37.621004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280290+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:06:18.280290+0000","last_peered":"2023-11-17T00:06:18.280290+0000","last_clean":"2023-11-17T00:06:18.280290+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:06:18.280290+0000","last_undegraded":"2023-11-17T00:06:18.280290+0000","last_fullsized":"2023-11-17T00:06:18.280290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:40.461673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214959+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:06:18.214959+0000","last_peered":"2023-11-17T00:06:18.214959+0000","last_clean":"2023-11-17T00:06:18.214959+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:06:18.214959+0000","last_undegraded":"2023-11-17T00:06:18.214959+0000","last_fullsized":"2023-11-17T00:06:18.214959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:25.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606344+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:06:17.606344+0000","last_peered":"2023-11-17T00:06:17.606344+0000","last_clean":"2023-11-17T00:06:17.606344+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:06:17.606344+0000","last_undegraded":"2023-11-17T00:06:17.606344+0000","last_fullsized":"2023-11-17T00:06:17.606344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:06:18.197799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215651+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:06:18.215651+0000","last_peered":"2023-11-17T00:06:18.215651+0000","last_clean":"2023-11-17T00:06:18.215651+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:06:18.215651+0000","last_undegraded":"2023-11-17T00:06:18.215651+0000","last_fullsized":"2023-11-17T00:06:18.215651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:49:47.472190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.213879+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:06:18.213879+0000","last_peered":"2023-11-17T00:06:18.213879+0000","last_clean":"2023-11-17T00:06:18.213879+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:06:18.213879+0000","last_undegraded":"2023-11-17T00:06:18.213879+0000","last_fullsized":"2023-11-17T00:06:18.213879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:25:47.541948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606797+0000","last_change":"2023-11-17T00:06:12.099094+0000","last_active":"2023-11-17T00:06:17.606797+0000","last_peered":"2023-11-17T00:06:17.606797+0000","last_clean":"2023-11-17T00:06:17.606797+0000","last_became_active":"2023-11-17T00:06:12.098773+0000","last_became_peered":"2023-11-17T00:06:12.098773+0000","last_unstale":"2023-11-17T00:06:17.606797+0000","last_undegraded":"2023-11-17T00:06:17.606797+0000","last_fullsized":"2023-11-17T00:06:17.606797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:57:24.076411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606309+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:06:17.606309+0000","last_peered":"2023-11-17T00:06:17.606309+0000","last_clean":"2023-11-17T00:06:17.606309+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:06:17.606309+0000","last_undegraded":"2023-11-17T00:06:17.606309+0000","last_fullsized":"2023-11-17T00:06:17.606309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:14:04.521227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215603+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:06:18.215603+0000","last_peered":"2023-11-17T00:06:18.215603+0000","last_clean":"2023-11-17T00:06:18.215603+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:06:18.215603+0000","last_undegraded":"2023-11-17T00:06:18.215603+0000","last_fullsized":"2023-11-17T00:06:18.215603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:14.199388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214520+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:06:18.214520+0000","last_peered":"2023-11-17T00:06:18.214520+0000","last_clean":"2023-11-17T00:06:18.214520+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:06:18.214520+0000","last_undegraded":"2023-11-17T00:06:18.214520+0000","last_fullsized":"2023-11-17T00:06:18.214520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:55:49.395655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280333+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:06:18.280333+0000","last_peered":"2023-11-17T00:06:18.280333+0000","last_clean":"2023-11-17T00:06:18.280333+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:06:18.280333+0000","last_undegraded":"2023-11-17T00:06:18.280333+0000","last_fullsized":"2023-11-17T00:06:18.280333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:00:45.033794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014894+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:06:18.014894+0000","last_peered":"2023-11-17T00:06:18.014894+0000","last_clean":"2023-11-17T00:06:18.014894+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:06:18.014894+0000","last_undegraded":"2023-11-17T00:06:18.014894+0000","last_fullsized":"2023-11-17T00:06:18.014894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:38.210721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281034+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:06:18.281034+0000","last_peered":"2023-11-17T00:06:18.281034+0000","last_clean":"2023-11-17T00:06:18.281034+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:06:18.281034+0000","last_undegraded":"2023-11-17T00:06:18.281034+0000","last_fullsized":"2023-11-17T00:06:18.281034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:26:38.537602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215857+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:06:18.215857+0000","last_peered":"2023-11-17T00:06:18.215857+0000","last_clean":"2023-11-17T00:06:18.215857+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:06:18.215857+0000","last_undegraded":"2023-11-17T00:06:18.215857+0000","last_fullsized":"2023-11-17T00:06:18.215857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:18.086346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214557+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:06:18.214557+0000","last_peered":"2023-11-17T00:06:18.214557+0000","last_clean":"2023-11-17T00:06:18.214557+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:06:18.214557+0000","last_undegraded":"2023-11-17T00:06:18.214557+0000","last_fullsized":"2023-11-17T00:06:18.214557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:14.825594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606156+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:06:17.606156+0000","last_peered":"2023-11-17T00:06:17.606156+0000","last_clean":"2023-11-17T00:06:17.606156+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:06:17.606156+0000","last_undegraded":"2023-11-17T00:06:17.606156+0000","last_fullsized":"2023-11-17T00:06:17.606156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:58:48.317787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214548+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:06:18.214548+0000","last_peered":"2023-11-17T00:06:18.214548+0000","last_clean":"2023-11-17T00:06:18.214548+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:06:18.214548+0000","last_undegraded":"2023-11-17T00:06:18.214548+0000","last_fullsized":"2023-11-17T00:06:18.214548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:41:30.640462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216163+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:06:18.216163+0000","last_peered":"2023-11-17T00:06:18.216163+0000","last_clean":"2023-11-17T00:06:18.216163+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:06:18.216163+0000","last_undegraded":"2023-11-17T00:06:18.216163+0000","last_fullsized":"2023-11-17T00:06:18.216163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:02:47.147602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214409+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:06:18.214409+0000","last_peered":"2023-11-17T00:06:18.214409+0000","last_clean":"2023-11-17T00:06:18.214409+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:06:18.214409+0000","last_undegraded":"2023-11-17T00:06:18.214409+0000","last_fullsized":"2023-11-17T00:06:18.214409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:53:28.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.015000+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:06:18.015000+0000","last_peered":"2023-11-17T00:06:18.015000+0000","last_clean":"2023-11-17T00:06:18.015000+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:06:18.015000+0000","last_undegraded":"2023-11-17T00:06:18.015000+0000","last_fullsized":"2023-11-17T00:06:18.015000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:11:04.636961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280179+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:06:18.280179+0000","last_peered":"2023-11-17T00:06:18.280179+0000","last_clean":"2023-11-17T00:06:18.280179+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:06:18.280179+0000","last_undegraded":"2023-11-17T00:06:18.280179+0000","last_fullsized":"2023-11-17T00:06:18.280179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:26:42.978100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606124+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:06:17.606124+0000","last_peered":"2023-11-17T00:06:17.606124+0000","last_clean":"2023-11-17T00:06:17.606124+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:06:17.606124+0000","last_undegraded":"2023-11-17T00:06:17.606124+0000","last_fullsized":"2023-11-17T00:06:17.606124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:01:26.678978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214479+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:06:18.214479+0000","last_peered":"2023-11-17T00:06:18.214479+0000","last_clean":"2023-11-17T00:06:18.214479+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:06:18.214479+0000","last_undegraded":"2023-11-17T00:06:18.214479+0000","last_fullsized":"2023-11-17T00:06:18.214479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:07:22.034685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216015+0000","last_change":"2023-11-17T00:06:12.106527+0000","last_active":"2023-11-17T00:06:18.216015+0000","last_peered":"2023-11-17T00:06:18.216015+0000","last_clean":"2023-11-17T00:06:18.216015+0000","last_became_active":"2023-11-17T00:06:12.105751+0000","last_became_peered":"2023-11-17T00:06:12.105751+0000","last_unstale":"2023-11-17T00:06:18.216015+0000","last_undegraded":"2023-11-17T00:06:18.216015+0000","last_fullsized":"2023-11-17T00:06:18.216015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:41:23.039048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280718+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:06:18.280718+0000","last_peered":"2023-11-17T00:06:18.280718+0000","last_clean":"2023-11-17T00:06:18.280718+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:06:18.280718+0000","last_undegraded":"2023-11-17T00:06:18.280718+0000","last_fullsized":"2023-11-17T00:06:18.280718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:22:56.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214538+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:06:18.214538+0000","last_peered":"2023-11-17T00:06:18.214538+0000","last_clean":"2023-11-17T00:06:18.214538+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:06:18.214538+0000","last_undegraded":"2023-11-17T00:06:18.214538+0000","last_fullsized":"2023-11-17T00:06:18.214538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:42:10.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215228+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:06:18.215228+0000","last_peered":"2023-11-17T00:06:18.215228+0000","last_clean":"2023-11-17T00:06:18.215228+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:06:18.215228+0000","last_undegraded":"2023-11-17T00:06:18.215228+0000","last_fullsized":"2023-11-17T00:06:18.215228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:49:55.478480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606695+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:06:17.606695+0000","last_peered":"2023-11-17T00:06:17.606695+0000","last_clean":"2023-11-17T00:06:17.606695+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:06:17.606695+0000","last_undegraded":"2023-11-17T00:06:17.606695+0000","last_fullsized":"2023-11-17T00:06:17.606695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:29:37.476775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215658+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:06:18.215658+0000","last_peered":"2023-11-17T00:06:18.215658+0000","last_clean":"2023-11-17T00:06:18.215658+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:06:18.215658+0000","last_undegraded":"2023-11-17T00:06:18.215658+0000","last_fullsized":"2023-11-17T00:06:18.215658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:21:48.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216363+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:06:18.216363+0000","last_peered":"2023-11-17T00:06:18.216363+0000","last_clean":"2023-11-17T00:06:18.216363+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:06:18.216363+0000","last_undegraded":"2023-11-17T00:06:18.216363+0000","last_fullsized":"2023-11-17T00:06:18.216363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:44:37.361655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606956+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:06:17.606956+0000","last_peered":"2023-11-17T00:06:17.606956+0000","last_clean":"2023-11-17T00:06:17.606956+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:06:17.606956+0000","last_undegraded":"2023-11-17T00:06:17.606956+0000","last_fullsized":"2023-11-17T00:06:17.606956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:37.278016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605731+0000","last_change":"2023-11-17T00:06:17.099552+0000","last_active":"2023-11-17T00:06:17.605731+0000","last_peered":"2023-11-17T00:06:17.605731+0000","last_clean":"2023-11-17T00:06:17.605731+0000","last_became_active":"2023-11-17T00:06:17.096199+0000","last_became_peered":"2023-11-17T00:06:17.096199+0000","last_unstale":"2023-11-17T00:06:17.605731+0000","last_undegraded":"2023-11-17T00:06:17.605731+0000","last_fullsized":"2023-11-17T00:06:17.605731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:34:28.643498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215916+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:06:18.215916+0000","last_peered":"2023-11-17T00:06:18.215916+0000","last_clean":"2023-11-17T00:06:18.215916+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:06:18.215916+0000","last_undegraded":"2023-11-17T00:06:18.215916+0000","last_fullsized":"2023-11-17T00:06:18.215916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:06.737122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606484+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:06:17.606484+0000","last_peered":"2023-11-17T00:06:17.606484+0000","last_clean":"2023-11-17T00:06:17.606484+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:06:17.606484+0000","last_undegraded":"2023-11-17T00:06:17.606484+0000","last_fullsized":"2023-11-17T00:06:17.606484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:01:27.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214756+0000","last_change":"2023-11-17T00:06:17.086637+0000","last_active":"2023-11-17T00:06:18.214756+0000","last_peered":"2023-11-17T00:06:18.214756+0000","last_clean":"2023-11-17T00:06:18.214756+0000","last_became_active":"2023-11-17T00:06:17.086230+0000","last_became_peered":"2023-11-17T00:06:17.086230+0000","last_unstale":"2023-11-17T00:06:18.214756+0000","last_undegraded":"2023-11-17T00:06:18.214756+0000","last_fullsized":"2023-11-17T00:06:18.214756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:01:08.928880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214511+0000","last_change":"2023-11-17T00:06:12.105778+0000","last_active":"2023-11-17T00:06:18.214511+0000","last_peered":"2023-11-17T00:06:18.214511+0000","last_clean":"2023-11-17T00:06:18.214511+0000","last_became_active":"2023-11-17T00:06:12.105448+0000","last_became_peered":"2023-11-17T00:06:12.105448+0000","last_unstale":"2023-11-17T00:06:18.214511+0000","last_undegraded":"2023-11-17T00:06:18.214511+0000","last_fullsized":"2023-11-17T00:06:18.214511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:29:39.965541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606952+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:06:17.606952+0000","last_peered":"2023-11-17T00:06:17.606952+0000","last_clean":"2023-11-17T00:06:17.606952+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:06:17.606952+0000","last_undegraded":"2023-11-17T00:06:17.606952+0000","last_fullsized":"2023-11-17T00:06:17.606952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:26.318517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606203+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:06:17.606203+0000","last_peered":"2023-11-17T00:06:17.606203+0000","last_clean":"2023-11-17T00:06:17.606203+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:06:17.606203+0000","last_undegraded":"2023-11-17T00:06:17.606203+0000","last_fullsized":"2023-11-17T00:06:17.606203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:47:08.134768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216431+0000","last_change":"2023-11-17T00:06:12.626187+0000","last_active":"2023-11-17T00:06:18.216431+0000","last_peered":"2023-11-17T00:06:18.216431+0000","last_clean":"2023-11-17T00:06:18.216431+0000","last_became_active":"2023-11-17T00:06:12.625849+0000","last_became_peered":"2023-11-17T00:06:12.625849+0000","last_unstale":"2023-11-17T00:06:18.216431+0000","last_undegraded":"2023-11-17T00:06:18.216431+0000","last_fullsized":"2023-11-17T00:06:18.216431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:41:21.076343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215878+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:06:18.215878+0000","last_peered":"2023-11-17T00:06:18.215878+0000","last_clean":"2023-11-17T00:06:18.215878+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:06:18.215878+0000","last_undegraded":"2023-11-17T00:06:18.215878+0000","last_fullsized":"2023-11-17T00:06:18.215878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:44.487229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605964+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:06:17.605964+0000","last_peered":"2023-11-17T00:06:17.605964+0000","last_clean":"2023-11-17T00:06:17.605964+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:06:17.605964+0000","last_undegraded":"2023-11-17T00:06:17.605964+0000","last_fullsized":"2023-11-17T00:06:17.605964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:09:25.079196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214607+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:06:18.214607+0000","last_peered":"2023-11-17T00:06:18.214607+0000","last_clean":"2023-11-17T00:06:18.214607+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:06:18.214607+0000","last_undegraded":"2023-11-17T00:06:18.214607+0000","last_fullsized":"2023-11-17T00:06:18.214607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:16.798137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606481+0000","last_change":"2023-11-17T00:06:12.638454+0000","last_active":"2023-11-17T00:06:17.606481+0000","last_peered":"2023-11-17T00:06:17.606481+0000","last_clean":"2023-11-17T00:06:17.606481+0000","last_became_active":"2023-11-17T00:06:12.637937+0000","last_became_peered":"2023-11-17T00:06:12.637937+0000","last_unstale":"2023-11-17T00:06:17.606481+0000","last_undegraded":"2023-11-17T00:06:17.606481+0000","last_fullsized":"2023-11-17T00:06:17.606481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:33:24.829924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216661+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:06:18.216661+0000","last_peered":"2023-11-17T00:06:18.216661+0000","last_clean":"2023-11-17T00:06:18.216661+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:06:18.216661+0000","last_undegraded":"2023-11-17T00:06:18.216661+0000","last_fullsized":"2023-11-17T00:06:18.216661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:08:03.664205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281119+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:06:18.281119+0000","last_peered":"2023-11-17T00:06:18.281119+0000","last_clean":"2023-11-17T00:06:18.281119+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:06:18.281119+0000","last_undegraded":"2023-11-17T00:06:18.281119+0000","last_fullsized":"2023-11-17T00:06:18.281119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:15.125337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215063+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:06:18.215063+0000","last_peered":"2023-11-17T00:06:18.215063+0000","last_clean":"2023-11-17T00:06:18.215063+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:06:18.215063+0000","last_undegraded":"2023-11-17T00:06:18.215063+0000","last_fullsized":"2023-11-17T00:06:18.215063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:53:32.592597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280322+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:06:18.280322+0000","last_peered":"2023-11-17T00:06:18.280322+0000","last_clean":"2023-11-17T00:06:18.280322+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:06:18.280322+0000","last_undegraded":"2023-11-17T00:06:18.280322+0000","last_fullsized":"2023-11-17T00:06:18.280322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:37:45.799364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215834+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:06:18.215834+0000","last_peered":"2023-11-17T00:06:18.215834+0000","last_clean":"2023-11-17T00:06:18.215834+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:06:18.215834+0000","last_undegraded":"2023-11-17T00:06:18.215834+0000","last_fullsized":"2023-11-17T00:06:18.215834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:22:26.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214231+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:06:18.214231+0000","last_peered":"2023-11-17T00:06:18.214231+0000","last_clean":"2023-11-17T00:06:18.214231+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:06:18.214231+0000","last_undegraded":"2023-11-17T00:06:18.214231+0000","last_fullsized":"2023-11-17T00:06:18.214231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:13:40.789671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606362+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:06:17.606362+0000","last_peered":"2023-11-17T00:06:17.606362+0000","last_clean":"2023-11-17T00:06:17.606362+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:06:17.606362+0000","last_undegraded":"2023-11-17T00:06:17.606362+0000","last_fullsized":"2023-11-17T00:06:17.606362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:30:10.134517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21973,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":12,"num_write":55,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":31157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6056,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739096,"statfs":{"total":95995035648,"available":95988834304,"internally_reserved":0,"allocated":950272,"data_stored":470966,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.38200000000000001}]},{"osd":1,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":4,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78000000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.747}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6400,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738752,"statfs":{"total":95995035648,"available":95988482048,"internally_reserved":0,"allocated":1040384,"data_stored":1082372,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.628}]},{"osd":6,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63700000000000001}]},{"osd":9,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.88},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74299999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6252,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738900,"statfs":{"total":95995035648,"available":95988633600,"internally_reserved":0,"allocated":954368,"data_stored":471548,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.504},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.624},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.64700000000000002}]},{"osd":8,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.877}]},{"osd":10,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.754},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":958464,"data_stored":471570,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.77800000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":1.139,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.075,"15min":1.075},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.745}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6492,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738660,"statfs":{"total":95995035648,"available":95988387840,"internally_reserved":0,"allocated":1003520,"data_stored":1061916,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.083,"15min":1.083},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82099999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.36399999999999999}]},{"osd":5,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.46999999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.046,"15min":1.046},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.173,"15min":1.173},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0429999999999999,"5min":1.129,"15min":1.129},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.40799999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.01,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55300000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.628}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6376,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738776,"statfs":{"total":95995035648,"available":95988506624,"internally_reserved":0,"allocated":950272,"data_stored":470966,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98699999999999999,"5min":1.02,"15min":1.02},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":1.206,"15min":1.206},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.006,"5min":1.262,"15min":1.262},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.038,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0860000000000001,"5min":1.294,"15min":1.294},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79700000000000004,"15min":0.754},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0509999999999999,"5min":1.544,"15min":1.544},"last":0.498},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.76200000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.377,"15min":1.377},"last":0.60999999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.234,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.628},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.749}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":962560,"data_stored":474760,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63}]},{"osd":2,"last update":"Fri Nov 17 00:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Nov 17 00:06:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60099999999999998,"15min":0.56799999999999995},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.433,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Nov 17 00:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.629},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93899999999999995,"5min":1.123,"15min":1.123},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.88300000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.55000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.872}]},{"osd":9,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85499999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.75}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7276,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737876,"statfs":{"total":95995035648,"available":95987585024,"internally_reserved":0,"allocated":954368,"data_stored":471056,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59599999999999997,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.62}]},{"osd":2,"last update":"Fri Nov 17 00:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.59499999999999997},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.871},{"interface":"front","average":{"1min":0.748,"5min":0.63600000000000001,"15min":0.58999999999999997},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 17 00:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95299999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63900000000000001,"15min":0.58199999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0660000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66500000000000004,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.374,"15min":0.374},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.67400000000000004,"15min":0.64900000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.98299999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94399999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.70699999999999996,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.755,"15min":0.74099999999999999},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.054,"5min":1.161,"15min":1.161},"last":0.745}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0209999999999999,"5min":1.377,"15min":1.377},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.022}]},{"osd":9,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":1.099},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97299999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.137}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":474794,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51400000000000001,"15min":0.47799999999999998},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.50700000000000001},"min":{"1min":0.42099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77300000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.754},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59699999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Nov 17 00:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63400000000000001,"15min":0.61499999999999999},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83299999999999996,"5min":0.875,"15min":0.875},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.51300000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64000000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.626},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.0069999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6636,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738516,"statfs":{"total":95995035648,"available":95988240384,"internally_reserved":0,"allocated":954368,"data_stored":470739,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.375,"5min":0.255,"15min":0.255},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.88,"15min":0.88},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.66700000000000004,"15min":0.63},"min":{"1min":0.53700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.008},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92800000000000005,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Nov 17 00:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.69799999999999995,"15min":0.66400000000000003},"min":{"1min":0.51500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.70699999999999996,"15min":0.67200000000000004},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Nov 17 00:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.131,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.58799999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.74199999999999999,"15min":0.71399999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6508,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738644,"statfs":{"total":95995035648,"available":95988371456,"internally_reserved":0,"allocated":954368,"data_stored":470988,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.29899999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54600000000000004,"15min":0.51600000000000001},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59199999999999997,"15min":0.53900000000000003},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85399999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Nov 17 00:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.67100000000000004,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.44,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.443,"15min":0.443},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6632,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738520,"statfs":{"total":95995035648,"available":95988244480,"internally_reserved":0,"allocated":1015808,"data_stored":1065612,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73599999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.437}]},{"osd":1,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.372},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83499999999999996,"5min":0.877,"15min":0.877},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0109999999999999,"5min":1.046,"15min":1.046},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.627},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65300000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":21369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:06:41.325 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: pgmap v307: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.0 KiB/s wr, 1 op/s 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3593564324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:40 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2400650488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:41.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.552+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/3046277314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b94101e50 msgr2=0x7f8b94108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.552+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/3046277314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b94101e50 0x7f8b94108710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b0a0 tx=0x7f8b8c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/3046277314 shutdown_connections 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/3046277314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b94101e50 0x7f8b94108710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/3046277314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b941002d0 0x7f8b941006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/3046277314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/3046277314 >> 172.21.15.6:0/3046277314 conn(0x7f8b940fb8b0 msgr2=0x7f8b940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/3046277314 shutdown_connections 2023-11-17T00:06:41.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.553+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/3046277314 wait complete. 2023-11-17T00:06:41.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 Processor -- start 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 -- start start 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b94101e50 0x7f8b9419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419dcd0 con 0x7f8b941002d0 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419de10 con 0x7f8b941014c0 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b9c1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419dfa0 con 0x7f8b94101e50 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.554+0000 7f8b99f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56324/0 (socket says 172.21.15.6:56324) 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b9979a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 -- 172.21.15.6:0/109850562 learned_addr learned my addr 172.21.15.6:0/109850562 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b9a79c700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b94101e50 0x7f8b9419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 -- 172.21.15.6:0/109850562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b94101e50 msgr2=0x7f8b9419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b94101e50 0x7f8b9419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 -- 172.21.15.6:0/109850562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 msgr2=0x7f8b94196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 -- 172.21.15.6:0/109850562 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009d00 con 0x7f8b941002d0 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b9979a700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:41.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b99f9b700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e970 tx=0x7f8b9000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.555+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90009790 con 0x7f8b941002d0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.556+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b9000c7e0 con 0x7f8b941002d0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.556+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9410bca0 con 0x7f8b941002d0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.556+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90003c70 con 0x7f8b941002d0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.556+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9410bb10 con 0x7f8b941002d0 2023-11-17T00:06:41.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.557+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b941077d0 con 0x7f8b941002d0 2023-11-17T00:06:41.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.560+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8b9000cc50 con 0x7f8b941002d0 2023-11-17T00:06:41.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.560+0000 7f8b8affd700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 0x7f8b800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:41.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.560+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b90014070 con 0x7f8b941002d0 2023-11-17T00:06:41.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.560+0000 7f8b9979a700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 0x7f8b800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:41.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.561+0000 7f8b9979a700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 0x7f8b800a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b84011490 tx=0x7f8b84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:41.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.562+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b90096350 con 0x7f8b941002d0 2023-11-17T00:06:41.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.767+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b940619b0 con 0x7f8b800a1260 2023-11-17T00:06:41.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.780+0000 7f8b8affd700 1 -- 172.21.15.6:0/109850562 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422028 (secure 0 0 0) 0x7f8b940619b0 con 0x7f8b800a1260 2023-11-17T00:06:41.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:41.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 msgr2=0x7f8b800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 0x7f8b800a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b84011490 tx=0x7f8b84009260 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 msgr2=0x7f8b94196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e970 tx=0x7f8b9000ec80 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 shutdown_connections 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8b800a1260 0x7f8b800a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b941002d0 0x7f8b94196150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b94101e50 0x7f8b9419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 --2- 172.21.15.6:0/109850562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b941014c0 0x7f8b94196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 >> 172.21.15.6:0/109850562 conn(0x7f8b940fb8b0 msgr2=0x7f8b94104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 shutdown_connections 2023-11-17T00:06:41.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:41.786+0000 7f8b9c1ff700 1 -- 172.21.15.6:0/109850562 wait complete. 2023-11-17T00:06:41.797 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:06:41.808 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-11-17T00:06:41.745064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612464,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":239,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79600,"kb_used_data":11408,"kb_used_omap":0,"kb_used_meta":68096,"kb_avail":1124862224,"statfs":{"total":1151940427776,"available":1151858917376,"internally_reserved":0,"allocated":11681792,"data_stored":7473671,"data_compressed":247614,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69730304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17645,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":13,"num_write":10,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003435"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606911+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:06:17.606911+0000","last_peered":"2023-11-17T00:06:17.606911+0000","last_clean":"2023-11-17T00:06:17.606911+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:06:17.606911+0000","last_undegraded":"2023-11-17T00:06:17.606911+0000","last_fullsized":"2023-11-17T00:06:17.606911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:59:51.597677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606065+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:06:17.606065+0000","last_peered":"2023-11-17T00:06:17.606065+0000","last_clean":"2023-11-17T00:06:17.606065+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:06:17.606065+0000","last_undegraded":"2023-11-17T00:06:17.606065+0000","last_fullsized":"2023-11-17T00:06:17.606065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:16:04.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605953+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:06:17.605953+0000","last_peered":"2023-11-17T00:06:17.605953+0000","last_clean":"2023-11-17T00:06:17.605953+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:06:17.605953+0000","last_undegraded":"2023-11-17T00:06:17.605953+0000","last_fullsized":"2023-11-17T00:06:17.605953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:34:09.527417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215164+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:06:18.215164+0000","last_peered":"2023-11-17T00:06:18.215164+0000","last_clean":"2023-11-17T00:06:18.215164+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:06:18.215164+0000","last_undegraded":"2023-11-17T00:06:18.215164+0000","last_fullsized":"2023-11-17T00:06:18.215164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:25.724124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214573+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:06:18.214573+0000","last_peered":"2023-11-17T00:06:18.214573+0000","last_clean":"2023-11-17T00:06:18.214573+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:06:18.214573+0000","last_undegraded":"2023-11-17T00:06:18.214573+0000","last_fullsized":"2023-11-17T00:06:18.214573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:59:56.856981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214611+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:06:18.214611+0000","last_peered":"2023-11-17T00:06:18.214611+0000","last_clean":"2023-11-17T00:06:18.214611+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:06:18.214611+0000","last_undegraded":"2023-11-17T00:06:18.214611+0000","last_fullsized":"2023-11-17T00:06:18.214611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:09:42.846932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606243+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:06:17.606243+0000","last_peered":"2023-11-17T00:06:17.606243+0000","last_clean":"2023-11-17T00:06:17.606243+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:06:17.606243+0000","last_undegraded":"2023-11-17T00:06:17.606243+0000","last_fullsized":"2023-11-17T00:06:17.606243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:39:59.072403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280390+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:06:18.280390+0000","last_peered":"2023-11-17T00:06:18.280390+0000","last_clean":"2023-11-17T00:06:18.280390+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:06:18.280390+0000","last_undegraded":"2023-11-17T00:06:18.280390+0000","last_fullsized":"2023-11-17T00:06:18.280390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:17:15.022762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216147+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:06:18.216147+0000","last_peered":"2023-11-17T00:06:18.216147+0000","last_clean":"2023-11-17T00:06:18.216147+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:06:18.216147+0000","last_undegraded":"2023-11-17T00:06:18.216147+0000","last_fullsized":"2023-11-17T00:06:18.216147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:06.166420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215489+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:06:18.215489+0000","last_peered":"2023-11-17T00:06:18.215489+0000","last_clean":"2023-11-17T00:06:18.215489+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:06:18.215489+0000","last_undegraded":"2023-11-17T00:06:18.215489+0000","last_fullsized":"2023-11-17T00:06:18.215489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:55:34.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606830+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:06:17.606830+0000","last_peered":"2023-11-17T00:06:17.606830+0000","last_clean":"2023-11-17T00:06:17.606830+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:06:17.606830+0000","last_undegraded":"2023-11-17T00:06:17.606830+0000","last_fullsized":"2023-11-17T00:06:17.606830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:48:01.654901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215022+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:06:18.215022+0000","last_peered":"2023-11-17T00:06:18.215022+0000","last_clean":"2023-11-17T00:06:18.215022+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:06:18.215022+0000","last_undegraded":"2023-11-17T00:06:18.215022+0000","last_fullsized":"2023-11-17T00:06:18.215022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:15.312936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214430+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:06:18.214430+0000","last_peered":"2023-11-17T00:06:18.214430+0000","last_clean":"2023-11-17T00:06:18.214430+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:06:18.214430+0000","last_undegraded":"2023-11-17T00:06:18.214430+0000","last_fullsized":"2023-11-17T00:06:18.214430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:31:08.188039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605613+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:06:17.605613+0000","last_peered":"2023-11-17T00:06:17.605613+0000","last_clean":"2023-11-17T00:06:17.605613+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:06:17.605613+0000","last_undegraded":"2023-11-17T00:06:17.605613+0000","last_fullsized":"2023-11-17T00:06:17.605613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:08:28.258146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605775+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:06:17.605775+0000","last_peered":"2023-11-17T00:06:17.605775+0000","last_clean":"2023-11-17T00:06:17.605775+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:06:17.605775+0000","last_undegraded":"2023-11-17T00:06:17.605775+0000","last_fullsized":"2023-11-17T00:06:17.605775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:44:47.871198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216141+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:06:18.216141+0000","last_peered":"2023-11-17T00:06:18.216141+0000","last_clean":"2023-11-17T00:06:18.216141+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:06:18.216141+0000","last_undegraded":"2023-11-17T00:06:18.216141+0000","last_fullsized":"2023-11-17T00:06:18.216141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:56:05.011055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606750+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:06:17.606750+0000","last_peered":"2023-11-17T00:06:17.606750+0000","last_clean":"2023-11-17T00:06:17.606750+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:06:17.606750+0000","last_undegraded":"2023-11-17T00:06:17.606750+0000","last_fullsized":"2023-11-17T00:06:17.606750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:49:56.168140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014883+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:06:18.014883+0000","last_peered":"2023-11-17T00:06:18.014883+0000","last_clean":"2023-11-17T00:06:18.014883+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:06:18.014883+0000","last_undegraded":"2023-11-17T00:06:18.014883+0000","last_fullsized":"2023-11-17T00:06:18.014883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:04:11.558089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216547+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:06:18.216547+0000","last_peered":"2023-11-17T00:06:18.216547+0000","last_clean":"2023-11-17T00:06:18.216547+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:06:18.216547+0000","last_undegraded":"2023-11-17T00:06:18.216547+0000","last_fullsized":"2023-11-17T00:06:18.216547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:19:03.058159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216065+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:06:18.216065+0000","last_peered":"2023-11-17T00:06:18.216065+0000","last_clean":"2023-11-17T00:06:18.216065+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:06:18.216065+0000","last_undegraded":"2023-11-17T00:06:18.216065+0000","last_fullsized":"2023-11-17T00:06:18.216065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:46.436458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.015028+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:06:18.015028+0000","last_peered":"2023-11-17T00:06:18.015028+0000","last_clean":"2023-11-17T00:06:18.015028+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:06:18.015028+0000","last_undegraded":"2023-11-17T00:06:18.015028+0000","last_fullsized":"2023-11-17T00:06:18.015028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:46:03.054290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280846+0000","last_change":"2023-11-17T00:06:12.628519+0000","last_active":"2023-11-17T00:06:18.280846+0000","last_peered":"2023-11-17T00:06:18.280846+0000","last_clean":"2023-11-17T00:06:18.280846+0000","last_became_active":"2023-11-17T00:06:12.627360+0000","last_became_peered":"2023-11-17T00:06:12.627360+0000","last_unstale":"2023-11-17T00:06:18.280846+0000","last_undegraded":"2023-11-17T00:06:18.280846+0000","last_fullsized":"2023-11-17T00:06:18.280846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:45:56.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214319+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:06:18.214319+0000","last_peered":"2023-11-17T00:06:18.214319+0000","last_clean":"2023-11-17T00:06:18.214319+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:06:18.214319+0000","last_undegraded":"2023-11-17T00:06:18.214319+0000","last_fullsized":"2023-11-17T00:06:18.214319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:27:48.580204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281003+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:06:18.281003+0000","last_peered":"2023-11-17T00:06:18.281003+0000","last_clean":"2023-11-17T00:06:18.281003+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:06:18.281003+0000","last_undegraded":"2023-11-17T00:06:18.281003+0000","last_fullsized":"2023-11-17T00:06:18.281003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:03:47.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606195+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:06:17.606195+0000","last_peered":"2023-11-17T00:06:17.606195+0000","last_clean":"2023-11-17T00:06:17.606195+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:06:17.606195+0000","last_undegraded":"2023-11-17T00:06:17.606195+0000","last_fullsized":"2023-11-17T00:06:17.606195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:28:19.627128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214931+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:06:18.214931+0000","last_peered":"2023-11-17T00:06:18.214931+0000","last_clean":"2023-11-17T00:06:18.214931+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:06:18.214931+0000","last_undegraded":"2023-11-17T00:06:18.214931+0000","last_fullsized":"2023-11-17T00:06:18.214931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:48:29.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215799+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:06:18.215799+0000","last_peered":"2023-11-17T00:06:18.215799+0000","last_clean":"2023-11-17T00:06:18.215799+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:06:18.215799+0000","last_undegraded":"2023-11-17T00:06:18.215799+0000","last_fullsized":"2023-11-17T00:06:18.215799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:35.530902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215028+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:06:18.215028+0000","last_peered":"2023-11-17T00:06:18.215028+0000","last_clean":"2023-11-17T00:06:18.215028+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:06:18.215028+0000","last_undegraded":"2023-11-17T00:06:18.215028+0000","last_fullsized":"2023-11-17T00:06:18.215028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:56.991464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214632+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:06:18.214632+0000","last_peered":"2023-11-17T00:06:18.214632+0000","last_clean":"2023-11-17T00:06:18.214632+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:06:18.214632+0000","last_undegraded":"2023-11-17T00:06:18.214632+0000","last_fullsized":"2023-11-17T00:06:18.214632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:33:01.556713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215066+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:06:18.215066+0000","last_peered":"2023-11-17T00:06:18.215066+0000","last_clean":"2023-11-17T00:06:18.215066+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:06:18.215066+0000","last_undegraded":"2023-11-17T00:06:18.215066+0000","last_fullsized":"2023-11-17T00:06:18.215066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:25:59.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216253+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:06:18.216253+0000","last_peered":"2023-11-17T00:06:18.216253+0000","last_clean":"2023-11-17T00:06:18.216253+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:06:18.216253+0000","last_undegraded":"2023-11-17T00:06:18.216253+0000","last_fullsized":"2023-11-17T00:06:18.216253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:57:51.933886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216310+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:06:18.216310+0000","last_peered":"2023-11-17T00:06:18.216310+0000","last_clean":"2023-11-17T00:06:18.216310+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:06:18.216310+0000","last_undegraded":"2023-11-17T00:06:18.216310+0000","last_fullsized":"2023-11-17T00:06:18.216310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:01.254910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215196+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:06:18.215196+0000","last_peered":"2023-11-17T00:06:18.215196+0000","last_clean":"2023-11-17T00:06:18.215196+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:06:18.215196+0000","last_undegraded":"2023-11-17T00:06:18.215196+0000","last_fullsized":"2023-11-17T00:06:18.215196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:39:02.495874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215163+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:06:18.215163+0000","last_peered":"2023-11-17T00:06:18.215163+0000","last_clean":"2023-11-17T00:06:18.215163+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:06:18.215163+0000","last_undegraded":"2023-11-17T00:06:18.215163+0000","last_fullsized":"2023-11-17T00:06:18.215163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:40:11.905147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216186+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:06:18.216186+0000","last_peered":"2023-11-17T00:06:18.216186+0000","last_clean":"2023-11-17T00:06:18.216186+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:06:18.216186+0000","last_undegraded":"2023-11-17T00:06:18.216186+0000","last_fullsized":"2023-11-17T00:06:18.216186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:45:47.937434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214558+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:06:18.214558+0000","last_peered":"2023-11-17T00:06:18.214558+0000","last_clean":"2023-11-17T00:06:18.214558+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:06:18.214558+0000","last_undegraded":"2023-11-17T00:06:18.214558+0000","last_fullsized":"2023-11-17T00:06:18.214558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:18:01.928364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215417+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:06:18.215417+0000","last_peered":"2023-11-17T00:06:18.215417+0000","last_clean":"2023-11-17T00:06:18.215417+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:06:18.215417+0000","last_undegraded":"2023-11-17T00:06:18.215417+0000","last_fullsized":"2023-11-17T00:06:18.215417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:28:49.567187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606751+0000","last_change":"2023-11-17T00:06:17.102667+0000","last_active":"2023-11-17T00:06:17.606751+0000","last_peered":"2023-11-17T00:06:17.606751+0000","last_clean":"2023-11-17T00:06:17.606751+0000","last_became_active":"2023-11-17T00:06:17.102273+0000","last_became_peered":"2023-11-17T00:06:17.102273+0000","last_unstale":"2023-11-17T00:06:17.606751+0000","last_undegraded":"2023-11-17T00:06:17.606751+0000","last_fullsized":"2023-11-17T00:06:17.606751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:06:18.674176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214070+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:06:18.214070+0000","last_peered":"2023-11-17T00:06:18.214070+0000","last_clean":"2023-11-17T00:06:18.214070+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:06:18.214070+0000","last_undegraded":"2023-11-17T00:06:18.214070+0000","last_fullsized":"2023-11-17T00:06:18.214070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:08:51.997009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280220+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:06:18.280220+0000","last_peered":"2023-11-17T00:06:18.280220+0000","last_clean":"2023-11-17T00:06:18.280220+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:06:18.280220+0000","last_undegraded":"2023-11-17T00:06:18.280220+0000","last_fullsized":"2023-11-17T00:06:18.280220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:18:24.283794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606112+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:06:17.606112+0000","last_peered":"2023-11-17T00:06:17.606112+0000","last_clean":"2023-11-17T00:06:17.606112+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:06:17.606112+0000","last_undegraded":"2023-11-17T00:06:17.606112+0000","last_fullsized":"2023-11-17T00:06:17.606112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:30:08.247199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214924+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:06:18.214924+0000","last_peered":"2023-11-17T00:06:18.214924+0000","last_clean":"2023-11-17T00:06:18.214924+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:06:18.214924+0000","last_undegraded":"2023-11-17T00:06:18.214924+0000","last_fullsized":"2023-11-17T00:06:18.214924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:30:02.850350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014324+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:06:18.014324+0000","last_peered":"2023-11-17T00:06:18.014324+0000","last_clean":"2023-11-17T00:06:18.014324+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:06:18.014324+0000","last_undegraded":"2023-11-17T00:06:18.014324+0000","last_fullsized":"2023-11-17T00:06:18.014324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:13:23.416055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280538+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:06:18.280538+0000","last_peered":"2023-11-17T00:06:18.280538+0000","last_clean":"2023-11-17T00:06:18.280538+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:06:18.280538+0000","last_undegraded":"2023-11-17T00:06:18.280538+0000","last_fullsized":"2023-11-17T00:06:18.280538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:38:35.251669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215393+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:06:18.215393+0000","last_peered":"2023-11-17T00:06:18.215393+0000","last_clean":"2023-11-17T00:06:18.215393+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:06:18.215393+0000","last_undegraded":"2023-11-17T00:06:18.215393+0000","last_fullsized":"2023-11-17T00:06:18.215393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:36:09.826273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216005+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:06:18.216005+0000","last_peered":"2023-11-17T00:06:18.216005+0000","last_clean":"2023-11-17T00:06:18.216005+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:06:18.216005+0000","last_undegraded":"2023-11-17T00:06:18.216005+0000","last_fullsized":"2023-11-17T00:06:18.216005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:29:15.971648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014232+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:06:18.014232+0000","last_peered":"2023-11-17T00:06:18.014232+0000","last_clean":"2023-11-17T00:06:18.014232+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:06:18.014232+0000","last_undegraded":"2023-11-17T00:06:18.014232+0000","last_fullsized":"2023-11-17T00:06:18.014232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:44:46.704510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606766+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:06:17.606766+0000","last_peered":"2023-11-17T00:06:17.606766+0000","last_clean":"2023-11-17T00:06:17.606766+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:06:17.606766+0000","last_undegraded":"2023-11-17T00:06:17.606766+0000","last_fullsized":"2023-11-17T00:06:17.606766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:30:05.357211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606209+0000","last_change":"2023-11-17T00:06:12.638172+0000","last_active":"2023-11-17T00:06:17.606209+0000","last_peered":"2023-11-17T00:06:17.606209+0000","last_clean":"2023-11-17T00:06:17.606209+0000","last_became_active":"2023-11-17T00:06:12.637227+0000","last_became_peered":"2023-11-17T00:06:12.637227+0000","last_unstale":"2023-11-17T00:06:17.606209+0000","last_undegraded":"2023-11-17T00:06:17.606209+0000","last_fullsized":"2023-11-17T00:06:17.606209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:44:00.879902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215124+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:06:18.215124+0000","last_peered":"2023-11-17T00:06:18.215124+0000","last_clean":"2023-11-17T00:06:18.215124+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:06:18.215124+0000","last_undegraded":"2023-11-17T00:06:18.215124+0000","last_fullsized":"2023-11-17T00:06:18.215124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:50.709860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215696+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:06:18.215696+0000","last_peered":"2023-11-17T00:06:18.215696+0000","last_clean":"2023-11-17T00:06:18.215696+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:06:18.215696+0000","last_undegraded":"2023-11-17T00:06:18.215696+0000","last_fullsized":"2023-11-17T00:06:18.215696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:56:21.913956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215253+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:06:18.215253+0000","last_peered":"2023-11-17T00:06:18.215253+0000","last_clean":"2023-11-17T00:06:18.215253+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:06:18.215253+0000","last_undegraded":"2023-11-17T00:06:18.215253+0000","last_fullsized":"2023-11-17T00:06:18.215253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:34:55.155458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014343+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:06:18.014343+0000","last_peered":"2023-11-17T00:06:18.014343+0000","last_clean":"2023-11-17T00:06:18.014343+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:06:18.014343+0000","last_undegraded":"2023-11-17T00:06:18.014343+0000","last_fullsized":"2023-11-17T00:06:18.014343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:51.824064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'16","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:38.361186+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:06:38.361186+0000","last_peered":"2023-11-17T00:06:38.361186+0000","last_clean":"2023-11-17T00:06:38.361186+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:06:38.361186+0000","last_undegraded":"2023-11-17T00:06:38.361186+0000","last_fullsized":"2023-11-17T00:06:38.361186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:28:44.761111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21175,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":16,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215470+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:06:18.215470+0000","last_peered":"2023-11-17T00:06:18.215470+0000","last_clean":"2023-11-17T00:06:18.215470+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:06:18.215470+0000","last_undegraded":"2023-11-17T00:06:18.215470+0000","last_fullsized":"2023-11-17T00:06:18.215470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:16.899631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216226+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:06:18.216226+0000","last_peered":"2023-11-17T00:06:18.216226+0000","last_clean":"2023-11-17T00:06:18.216226+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:06:18.216226+0000","last_undegraded":"2023-11-17T00:06:18.216226+0000","last_fullsized":"2023-11-17T00:06:18.216226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:16:56.547530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216332+0000","last_change":"2023-11-17T00:06:12.107382+0000","last_active":"2023-11-17T00:06:18.216332+0000","last_peered":"2023-11-17T00:06:18.216332+0000","last_clean":"2023-11-17T00:06:18.216332+0000","last_became_active":"2023-11-17T00:06:12.105773+0000","last_became_peered":"2023-11-17T00:06:12.105773+0000","last_unstale":"2023-11-17T00:06:18.216332+0000","last_undegraded":"2023-11-17T00:06:18.216332+0000","last_fullsized":"2023-11-17T00:06:18.216332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:18.065738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214106+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:06:18.214106+0000","last_peered":"2023-11-17T00:06:18.214106+0000","last_clean":"2023-11-17T00:06:18.214106+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:06:18.214106+0000","last_undegraded":"2023-11-17T00:06:18.214106+0000","last_fullsized":"2023-11-17T00:06:18.214106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:09:28.039505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606311+0000","last_change":"2023-11-17T00:06:17.104407+0000","last_active":"2023-11-17T00:06:17.606311+0000","last_peered":"2023-11-17T00:06:17.606311+0000","last_clean":"2023-11-17T00:06:17.606311+0000","last_became_active":"2023-11-17T00:06:17.103089+0000","last_became_peered":"2023-11-17T00:06:17.103089+0000","last_unstale":"2023-11-17T00:06:17.606311+0000","last_undegraded":"2023-11-17T00:06:17.606311+0000","last_fullsized":"2023-11-17T00:06:17.606311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:45:58.949245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215544+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:06:18.215544+0000","last_peered":"2023-11-17T00:06:18.215544+0000","last_clean":"2023-11-17T00:06:18.215544+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:06:18.215544+0000","last_undegraded":"2023-11-17T00:06:18.215544+0000","last_fullsized":"2023-11-17T00:06:18.215544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:03:32.099745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606663+0000","last_change":"2023-11-17T00:06:12.632671+0000","last_active":"2023-11-17T00:06:17.606663+0000","last_peered":"2023-11-17T00:06:17.606663+0000","last_clean":"2023-11-17T00:06:17.606663+0000","last_became_active":"2023-11-17T00:06:12.632332+0000","last_became_peered":"2023-11-17T00:06:12.632332+0000","last_unstale":"2023-11-17T00:06:17.606663+0000","last_undegraded":"2023-11-17T00:06:17.606663+0000","last_fullsized":"2023-11-17T00:06:17.606663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:17.846720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014853+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:06:18.014853+0000","last_peered":"2023-11-17T00:06:18.014853+0000","last_clean":"2023-11-17T00:06:18.014853+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:06:18.014853+0000","last_undegraded":"2023-11-17T00:06:18.014853+0000","last_fullsized":"2023-11-17T00:06:18.014853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:22:02.039903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215789+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:06:18.215789+0000","last_peered":"2023-11-17T00:06:18.215789+0000","last_clean":"2023-11-17T00:06:18.215789+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:06:18.215789+0000","last_undegraded":"2023-11-17T00:06:18.215789+0000","last_fullsized":"2023-11-17T00:06:18.215789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:09:34.331979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606969+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:06:17.606969+0000","last_peered":"2023-11-17T00:06:17.606969+0000","last_clean":"2023-11-17T00:06:17.606969+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:06:17.606969+0000","last_undegraded":"2023-11-17T00:06:17.606969+0000","last_fullsized":"2023-11-17T00:06:17.606969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:54:48.901609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214492+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:06:18.214492+0000","last_peered":"2023-11-17T00:06:18.214492+0000","last_clean":"2023-11-17T00:06:18.214492+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:06:18.214492+0000","last_undegraded":"2023-11-17T00:06:18.214492+0000","last_fullsized":"2023-11-17T00:06:18.214492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:27:54.479757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214354+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:06:18.214354+0000","last_peered":"2023-11-17T00:06:18.214354+0000","last_clean":"2023-11-17T00:06:18.214354+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:06:18.214354+0000","last_undegraded":"2023-11-17T00:06:18.214354+0000","last_fullsized":"2023-11-17T00:06:18.214354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:51.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214640+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:06:18.214640+0000","last_peered":"2023-11-17T00:06:18.214640+0000","last_clean":"2023-11-17T00:06:18.214640+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:06:18.214640+0000","last_undegraded":"2023-11-17T00:06:18.214640+0000","last_fullsized":"2023-11-17T00:06:18.214640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:19:10.256370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214565+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:06:18.214565+0000","last_peered":"2023-11-17T00:06:18.214565+0000","last_clean":"2023-11-17T00:06:18.214565+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:06:18.214565+0000","last_undegraded":"2023-11-17T00:06:18.214565+0000","last_fullsized":"2023-11-17T00:06:18.214565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:23:36.703118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215049+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:06:18.215049+0000","last_peered":"2023-11-17T00:06:18.215049+0000","last_clean":"2023-11-17T00:06:18.215049+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:06:18.215049+0000","last_undegraded":"2023-11-17T00:06:18.215049+0000","last_fullsized":"2023-11-17T00:06:18.215049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:04:44.010870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606687+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:06:17.606687+0000","last_peered":"2023-11-17T00:06:17.606687+0000","last_clean":"2023-11-17T00:06:17.606687+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:06:17.606687+0000","last_undegraded":"2023-11-17T00:06:17.606687+0000","last_fullsized":"2023-11-17T00:06:17.606687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:03:50.330931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214973+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:06:18.214973+0000","last_peered":"2023-11-17T00:06:18.214973+0000","last_clean":"2023-11-17T00:06:18.214973+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:06:18.214973+0000","last_undegraded":"2023-11-17T00:06:18.214973+0000","last_fullsized":"2023-11-17T00:06:18.214973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:19:37.010947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014642+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:06:18.014642+0000","last_peered":"2023-11-17T00:06:18.014642+0000","last_clean":"2023-11-17T00:06:18.014642+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:06:18.014642+0000","last_undegraded":"2023-11-17T00:06:18.014642+0000","last_fullsized":"2023-11-17T00:06:18.014642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:01:46.650139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215360+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:06:18.215360+0000","last_peered":"2023-11-17T00:06:18.215360+0000","last_clean":"2023-11-17T00:06:18.215360+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:06:18.215360+0000","last_undegraded":"2023-11-17T00:06:18.215360+0000","last_fullsized":"2023-11-17T00:06:18.215360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:57:54.487416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214288+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:06:18.214288+0000","last_peered":"2023-11-17T00:06:18.214288+0000","last_clean":"2023-11-17T00:06:18.214288+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:06:18.214288+0000","last_undegraded":"2023-11-17T00:06:18.214288+0000","last_fullsized":"2023-11-17T00:06:18.214288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:03:22.517337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606446+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:06:17.606446+0000","last_peered":"2023-11-17T00:06:17.606446+0000","last_clean":"2023-11-17T00:06:17.606446+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:06:17.606446+0000","last_undegraded":"2023-11-17T00:06:17.606446+0000","last_fullsized":"2023-11-17T00:06:17.606446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:46:19.855956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215139+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:06:18.215139+0000","last_peered":"2023-11-17T00:06:18.215139+0000","last_clean":"2023-11-17T00:06:18.215139+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:06:18.215139+0000","last_undegraded":"2023-11-17T00:06:18.215139+0000","last_fullsized":"2023-11-17T00:06:18.215139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:37:43.731363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214302+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:06:18.214302+0000","last_peered":"2023-11-17T00:06:18.214302+0000","last_clean":"2023-11-17T00:06:18.214302+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:06:18.214302+0000","last_undegraded":"2023-11-17T00:06:18.214302+0000","last_fullsized":"2023-11-17T00:06:18.214302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:47:36.720175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215201+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:06:18.215201+0000","last_peered":"2023-11-17T00:06:18.215201+0000","last_clean":"2023-11-17T00:06:18.215201+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:06:18.215201+0000","last_undegraded":"2023-11-17T00:06:18.215201+0000","last_fullsized":"2023-11-17T00:06:18.215201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:47:33.653002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.213879+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:06:18.213879+0000","last_peered":"2023-11-17T00:06:18.213879+0000","last_clean":"2023-11-17T00:06:18.213879+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:06:18.213879+0000","last_undegraded":"2023-11-17T00:06:18.213879+0000","last_fullsized":"2023-11-17T00:06:18.213879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:23:27.835317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:19.903503+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:06:19.903503+0000","last_peered":"2023-11-17T00:06:19.903503+0000","last_clean":"2023-11-17T00:06:19.903503+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:06:19.903503+0000","last_undegraded":"2023-11-17T00:06:19.903503+0000","last_fullsized":"2023-11-17T00:06:19.903503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:05:37.621004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280290+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:06:18.280290+0000","last_peered":"2023-11-17T00:06:18.280290+0000","last_clean":"2023-11-17T00:06:18.280290+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:06:18.280290+0000","last_undegraded":"2023-11-17T00:06:18.280290+0000","last_fullsized":"2023-11-17T00:06:18.280290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:40.461673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214959+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:06:18.214959+0000","last_peered":"2023-11-17T00:06:18.214959+0000","last_clean":"2023-11-17T00:06:18.214959+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:06:18.214959+0000","last_undegraded":"2023-11-17T00:06:18.214959+0000","last_fullsized":"2023-11-17T00:06:18.214959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:25.026085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606344+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:06:17.606344+0000","last_peered":"2023-11-17T00:06:17.606344+0000","last_clean":"2023-11-17T00:06:17.606344+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:06:17.606344+0000","last_undegraded":"2023-11-17T00:06:17.606344+0000","last_fullsized":"2023-11-17T00:06:17.606344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:06:18.197799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215651+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:06:18.215651+0000","last_peered":"2023-11-17T00:06:18.215651+0000","last_clean":"2023-11-17T00:06:18.215651+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:06:18.215651+0000","last_undegraded":"2023-11-17T00:06:18.215651+0000","last_fullsized":"2023-11-17T00:06:18.215651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:49:47.472190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.213879+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:06:18.213879+0000","last_peered":"2023-11-17T00:06:18.213879+0000","last_clean":"2023-11-17T00:06:18.213879+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:06:18.213879+0000","last_undegraded":"2023-11-17T00:06:18.213879+0000","last_fullsized":"2023-11-17T00:06:18.213879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:25:47.541948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606797+0000","last_change":"2023-11-17T00:06:12.099094+0000","last_active":"2023-11-17T00:06:17.606797+0000","last_peered":"2023-11-17T00:06:17.606797+0000","last_clean":"2023-11-17T00:06:17.606797+0000","last_became_active":"2023-11-17T00:06:12.098773+0000","last_became_peered":"2023-11-17T00:06:12.098773+0000","last_unstale":"2023-11-17T00:06:17.606797+0000","last_undegraded":"2023-11-17T00:06:17.606797+0000","last_fullsized":"2023-11-17T00:06:17.606797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:57:24.076411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606309+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:06:17.606309+0000","last_peered":"2023-11-17T00:06:17.606309+0000","last_clean":"2023-11-17T00:06:17.606309+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:06:17.606309+0000","last_undegraded":"2023-11-17T00:06:17.606309+0000","last_fullsized":"2023-11-17T00:06:17.606309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:14:04.521227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215603+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:06:18.215603+0000","last_peered":"2023-11-17T00:06:18.215603+0000","last_clean":"2023-11-17T00:06:18.215603+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:06:18.215603+0000","last_undegraded":"2023-11-17T00:06:18.215603+0000","last_fullsized":"2023-11-17T00:06:18.215603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:14.199388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214520+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:06:18.214520+0000","last_peered":"2023-11-17T00:06:18.214520+0000","last_clean":"2023-11-17T00:06:18.214520+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:06:18.214520+0000","last_undegraded":"2023-11-17T00:06:18.214520+0000","last_fullsized":"2023-11-17T00:06:18.214520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:55:49.395655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280333+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:06:18.280333+0000","last_peered":"2023-11-17T00:06:18.280333+0000","last_clean":"2023-11-17T00:06:18.280333+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:06:18.280333+0000","last_undegraded":"2023-11-17T00:06:18.280333+0000","last_fullsized":"2023-11-17T00:06:18.280333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:00:45.033794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.014944+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:06:18.014944+0000","last_peered":"2023-11-17T00:06:18.014944+0000","last_clean":"2023-11-17T00:06:18.014944+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:06:18.014944+0000","last_undegraded":"2023-11-17T00:06:18.014944+0000","last_fullsized":"2023-11-17T00:06:18.014944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:38.210721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281034+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:06:18.281034+0000","last_peered":"2023-11-17T00:06:18.281034+0000","last_clean":"2023-11-17T00:06:18.281034+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:06:18.281034+0000","last_undegraded":"2023-11-17T00:06:18.281034+0000","last_fullsized":"2023-11-17T00:06:18.281034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:26:38.537602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215857+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:06:18.215857+0000","last_peered":"2023-11-17T00:06:18.215857+0000","last_clean":"2023-11-17T00:06:18.215857+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:06:18.215857+0000","last_undegraded":"2023-11-17T00:06:18.215857+0000","last_fullsized":"2023-11-17T00:06:18.215857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:18.086346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214557+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:06:18.214557+0000","last_peered":"2023-11-17T00:06:18.214557+0000","last_clean":"2023-11-17T00:06:18.214557+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:06:18.214557+0000","last_undegraded":"2023-11-17T00:06:18.214557+0000","last_fullsized":"2023-11-17T00:06:18.214557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:14.825594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606156+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:06:17.606156+0000","last_peered":"2023-11-17T00:06:17.606156+0000","last_clean":"2023-11-17T00:06:17.606156+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:06:17.606156+0000","last_undegraded":"2023-11-17T00:06:17.606156+0000","last_fullsized":"2023-11-17T00:06:17.606156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:58:48.317787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214548+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:06:18.214548+0000","last_peered":"2023-11-17T00:06:18.214548+0000","last_clean":"2023-11-17T00:06:18.214548+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:06:18.214548+0000","last_undegraded":"2023-11-17T00:06:18.214548+0000","last_fullsized":"2023-11-17T00:06:18.214548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:41:30.640462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216163+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:06:18.216163+0000","last_peered":"2023-11-17T00:06:18.216163+0000","last_clean":"2023-11-17T00:06:18.216163+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:06:18.216163+0000","last_undegraded":"2023-11-17T00:06:18.216163+0000","last_fullsized":"2023-11-17T00:06:18.216163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:02:47.147602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214409+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:06:18.214409+0000","last_peered":"2023-11-17T00:06:18.214409+0000","last_clean":"2023-11-17T00:06:18.214409+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:06:18.214409+0000","last_undegraded":"2023-11-17T00:06:18.214409+0000","last_fullsized":"2023-11-17T00:06:18.214409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:53:28.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.015000+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:06:18.015000+0000","last_peered":"2023-11-17T00:06:18.015000+0000","last_clean":"2023-11-17T00:06:18.015000+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:06:18.015000+0000","last_undegraded":"2023-11-17T00:06:18.015000+0000","last_fullsized":"2023-11-17T00:06:18.015000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:11:04.636961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280179+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:06:18.280179+0000","last_peered":"2023-11-17T00:06:18.280179+0000","last_clean":"2023-11-17T00:06:18.280179+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:06:18.280179+0000","last_undegraded":"2023-11-17T00:06:18.280179+0000","last_fullsized":"2023-11-17T00:06:18.280179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:26:42.978100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606124+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:06:17.606124+0000","last_peered":"2023-11-17T00:06:17.606124+0000","last_clean":"2023-11-17T00:06:17.606124+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:06:17.606124+0000","last_undegraded":"2023-11-17T00:06:17.606124+0000","last_fullsized":"2023-11-17T00:06:17.606124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:01:26.678978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214479+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:06:18.214479+0000","last_peered":"2023-11-17T00:06:18.214479+0000","last_clean":"2023-11-17T00:06:18.214479+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:06:18.214479+0000","last_undegraded":"2023-11-17T00:06:18.214479+0000","last_fullsized":"2023-11-17T00:06:18.214479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:07:22.034685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216015+0000","last_change":"2023-11-17T00:06:12.106527+0000","last_active":"2023-11-17T00:06:18.216015+0000","last_peered":"2023-11-17T00:06:18.216015+0000","last_clean":"2023-11-17T00:06:18.216015+0000","last_became_active":"2023-11-17T00:06:12.105751+0000","last_became_peered":"2023-11-17T00:06:12.105751+0000","last_unstale":"2023-11-17T00:06:18.216015+0000","last_undegraded":"2023-11-17T00:06:18.216015+0000","last_fullsized":"2023-11-17T00:06:18.216015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:41:23.039048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280718+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:06:18.280718+0000","last_peered":"2023-11-17T00:06:18.280718+0000","last_clean":"2023-11-17T00:06:18.280718+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:06:18.280718+0000","last_undegraded":"2023-11-17T00:06:18.280718+0000","last_fullsized":"2023-11-17T00:06:18.280718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:22:56.593849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214538+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:06:18.214538+0000","last_peered":"2023-11-17T00:06:18.214538+0000","last_clean":"2023-11-17T00:06:18.214538+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:06:18.214538+0000","last_undegraded":"2023-11-17T00:06:18.214538+0000","last_fullsized":"2023-11-17T00:06:18.214538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:42:10.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215228+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:06:18.215228+0000","last_peered":"2023-11-17T00:06:18.215228+0000","last_clean":"2023-11-17T00:06:18.215228+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:06:18.215228+0000","last_undegraded":"2023-11-17T00:06:18.215228+0000","last_fullsized":"2023-11-17T00:06:18.215228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:49:55.478480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606695+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:06:17.606695+0000","last_peered":"2023-11-17T00:06:17.606695+0000","last_clean":"2023-11-17T00:06:17.606695+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:06:17.606695+0000","last_undegraded":"2023-11-17T00:06:17.606695+0000","last_fullsized":"2023-11-17T00:06:17.606695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:29:37.476775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215658+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:06:18.215658+0000","last_peered":"2023-11-17T00:06:18.215658+0000","last_clean":"2023-11-17T00:06:18.215658+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:06:18.215658+0000","last_undegraded":"2023-11-17T00:06:18.215658+0000","last_fullsized":"2023-11-17T00:06:18.215658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:21:48.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216363+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:06:18.216363+0000","last_peered":"2023-11-17T00:06:18.216363+0000","last_clean":"2023-11-17T00:06:18.216363+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:06:18.216363+0000","last_undegraded":"2023-11-17T00:06:18.216363+0000","last_fullsized":"2023-11-17T00:06:18.216363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:44:37.361655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606956+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:06:17.606956+0000","last_peered":"2023-11-17T00:06:17.606956+0000","last_clean":"2023-11-17T00:06:17.606956+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:06:17.606956+0000","last_undegraded":"2023-11-17T00:06:17.606956+0000","last_fullsized":"2023-11-17T00:06:17.606956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:37.278016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605731+0000","last_change":"2023-11-17T00:06:17.099552+0000","last_active":"2023-11-17T00:06:17.605731+0000","last_peered":"2023-11-17T00:06:17.605731+0000","last_clean":"2023-11-17T00:06:17.605731+0000","last_became_active":"2023-11-17T00:06:17.096199+0000","last_became_peered":"2023-11-17T00:06:17.096199+0000","last_unstale":"2023-11-17T00:06:17.605731+0000","last_undegraded":"2023-11-17T00:06:17.605731+0000","last_fullsized":"2023-11-17T00:06:17.605731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:34:28.643498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215916+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:06:18.215916+0000","last_peered":"2023-11-17T00:06:18.215916+0000","last_clean":"2023-11-17T00:06:18.215916+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:06:18.215916+0000","last_undegraded":"2023-11-17T00:06:18.215916+0000","last_fullsized":"2023-11-17T00:06:18.215916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:56:06.737122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606484+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:06:17.606484+0000","last_peered":"2023-11-17T00:06:17.606484+0000","last_clean":"2023-11-17T00:06:17.606484+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:06:17.606484+0000","last_undegraded":"2023-11-17T00:06:17.606484+0000","last_fullsized":"2023-11-17T00:06:17.606484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:01:27.865078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214756+0000","last_change":"2023-11-17T00:06:17.086637+0000","last_active":"2023-11-17T00:06:18.214756+0000","last_peered":"2023-11-17T00:06:18.214756+0000","last_clean":"2023-11-17T00:06:18.214756+0000","last_became_active":"2023-11-17T00:06:17.086230+0000","last_became_peered":"2023-11-17T00:06:17.086230+0000","last_unstale":"2023-11-17T00:06:18.214756+0000","last_undegraded":"2023-11-17T00:06:18.214756+0000","last_fullsized":"2023-11-17T00:06:18.214756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:01:08.928880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214511+0000","last_change":"2023-11-17T00:06:12.105778+0000","last_active":"2023-11-17T00:06:18.214511+0000","last_peered":"2023-11-17T00:06:18.214511+0000","last_clean":"2023-11-17T00:06:18.214511+0000","last_became_active":"2023-11-17T00:06:12.105448+0000","last_became_peered":"2023-11-17T00:06:12.105448+0000","last_unstale":"2023-11-17T00:06:18.214511+0000","last_undegraded":"2023-11-17T00:06:18.214511+0000","last_fullsized":"2023-11-17T00:06:18.214511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:29:39.965541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606952+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:06:17.606952+0000","last_peered":"2023-11-17T00:06:17.606952+0000","last_clean":"2023-11-17T00:06:17.606952+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:06:17.606952+0000","last_undegraded":"2023-11-17T00:06:17.606952+0000","last_fullsized":"2023-11-17T00:06:17.606952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:26.318517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606203+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:06:17.606203+0000","last_peered":"2023-11-17T00:06:17.606203+0000","last_clean":"2023-11-17T00:06:17.606203+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:06:17.606203+0000","last_undegraded":"2023-11-17T00:06:17.606203+0000","last_fullsized":"2023-11-17T00:06:17.606203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:47:08.134768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216431+0000","last_change":"2023-11-17T00:06:12.626187+0000","last_active":"2023-11-17T00:06:18.216431+0000","last_peered":"2023-11-17T00:06:18.216431+0000","last_clean":"2023-11-17T00:06:18.216431+0000","last_became_active":"2023-11-17T00:06:12.625849+0000","last_became_peered":"2023-11-17T00:06:12.625849+0000","last_unstale":"2023-11-17T00:06:18.216431+0000","last_undegraded":"2023-11-17T00:06:18.216431+0000","last_fullsized":"2023-11-17T00:06:18.216431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:41:21.076343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215878+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:06:18.215878+0000","last_peered":"2023-11-17T00:06:18.215878+0000","last_clean":"2023-11-17T00:06:18.215878+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:06:18.215878+0000","last_undegraded":"2023-11-17T00:06:18.215878+0000","last_fullsized":"2023-11-17T00:06:18.215878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:44.487229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.605964+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:06:17.605964+0000","last_peered":"2023-11-17T00:06:17.605964+0000","last_clean":"2023-11-17T00:06:17.605964+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:06:17.605964+0000","last_undegraded":"2023-11-17T00:06:17.605964+0000","last_fullsized":"2023-11-17T00:06:17.605964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:09:25.079196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214607+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:06:18.214607+0000","last_peered":"2023-11-17T00:06:18.214607+0000","last_clean":"2023-11-17T00:06:18.214607+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:06:18.214607+0000","last_undegraded":"2023-11-17T00:06:18.214607+0000","last_fullsized":"2023-11-17T00:06:18.214607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:16.798137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606481+0000","last_change":"2023-11-17T00:06:12.638454+0000","last_active":"2023-11-17T00:06:17.606481+0000","last_peered":"2023-11-17T00:06:17.606481+0000","last_clean":"2023-11-17T00:06:17.606481+0000","last_became_active":"2023-11-17T00:06:12.637937+0000","last_became_peered":"2023-11-17T00:06:12.637937+0000","last_unstale":"2023-11-17T00:06:17.606481+0000","last_undegraded":"2023-11-17T00:06:17.606481+0000","last_fullsized":"2023-11-17T00:06:17.606481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:33:24.829924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.216661+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:06:18.216661+0000","last_peered":"2023-11-17T00:06:18.216661+0000","last_clean":"2023-11-17T00:06:18.216661+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:06:18.216661+0000","last_undegraded":"2023-11-17T00:06:18.216661+0000","last_fullsized":"2023-11-17T00:06:18.216661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:08:03.664205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.281119+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:06:18.281119+0000","last_peered":"2023-11-17T00:06:18.281119+0000","last_clean":"2023-11-17T00:06:18.281119+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:06:18.281119+0000","last_undegraded":"2023-11-17T00:06:18.281119+0000","last_fullsized":"2023-11-17T00:06:18.281119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:15.125337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215063+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:06:18.215063+0000","last_peered":"2023-11-17T00:06:18.215063+0000","last_clean":"2023-11-17T00:06:18.215063+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:06:18.215063+0000","last_undegraded":"2023-11-17T00:06:18.215063+0000","last_fullsized":"2023-11-17T00:06:18.215063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:53:32.592597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.280322+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:06:18.280322+0000","last_peered":"2023-11-17T00:06:18.280322+0000","last_clean":"2023-11-17T00:06:18.280322+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:06:18.280322+0000","last_undegraded":"2023-11-17T00:06:18.280322+0000","last_fullsized":"2023-11-17T00:06:18.280322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:37:45.799364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.215834+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:06:18.215834+0000","last_peered":"2023-11-17T00:06:18.215834+0000","last_clean":"2023-11-17T00:06:18.215834+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:06:18.215834+0000","last_undegraded":"2023-11-17T00:06:18.215834+0000","last_fullsized":"2023-11-17T00:06:18.215834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:22:26.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:18.214231+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:06:18.214231+0000","last_peered":"2023-11-17T00:06:18.214231+0000","last_clean":"2023-11-17T00:06:18.214231+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:06:18.214231+0000","last_undegraded":"2023-11-17T00:06:18.214231+0000","last_fullsized":"2023-11-17T00:06:18.214231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:13:40.789671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-17T00:06:17.606362+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:06:17.606362+0000","last_peered":"2023-11-17T00:06:17.606362+0000","last_clean":"2023-11-17T00:06:17.606362+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:06:17.606362+0000","last_undegraded":"2023-11-17T00:06:17.606362+0000","last_fullsized":"2023-11-17T00:06:17.606362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:30:10.134517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21973,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":13,"num_write":57,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":31157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6060,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739092,"statfs":{"total":95995035648,"available":95988830208,"internally_reserved":0,"allocated":954368,"data_stored":475062,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.502}]},{"osd":1,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":4,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77700000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":9,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.75600000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6400,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738752,"statfs":{"total":95995035648,"available":95988482048,"internally_reserved":0,"allocated":1040384,"data_stored":1082372,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.628}]},{"osd":6,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63700000000000001}]},{"osd":9,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.88},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74299999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6260,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738892,"statfs":{"total":95995035648,"available":95988625408,"internally_reserved":0,"allocated":962560,"data_stored":479740,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64400000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.437}]},{"osd":1,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83099999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.80600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84799999999999998,"5min":1.093,"15min":1.093},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.77600000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.224,"15min":1.224},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93999999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93400000000000005,"5min":1.28,"15min":1.28},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.85199999999999998,"15min":0.87},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.93100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0649999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.92200000000000004,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0900000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.66400000000000003}]},{"osd":8,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.98299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":1.004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.01,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":958464,"data_stored":471570,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84499999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.77800000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":1.139,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":1.075,"15min":1.075},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.71399999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.745}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6496,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738656,"statfs":{"total":95995035648,"available":95988383744,"internally_reserved":0,"allocated":1007616,"data_stored":1066012,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45200000000000001,"15min":0.45800000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.083,"15min":1.083},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82099999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.46999999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.046,"15min":1.046},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.173,"15min":1.173},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0429999999999999,"5min":1.129,"15min":1.129},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.78300000000000003}]},{"osd":8,"last update":"Fri Nov 17 00:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.01,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77300000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91000000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Fri Nov 17 00:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6376,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738776,"statfs":{"total":95995035648,"available":95988506624,"internally_reserved":0,"allocated":950272,"data_stored":470966,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81899999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.68899999999999995},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98699999999999999,"5min":1.02,"15min":1.02},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85799999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":1.206,"15min":1.206},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.006,"5min":1.262,"15min":1.262},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Nov 17 00:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.038,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0860000000000001,"5min":1.294,"15min":1.294},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79700000000000004,"15min":0.754},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0509999999999999,"5min":1.544,"15min":1.544},"last":0.498},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.76200000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.377,"15min":1.377},"last":0.60999999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.234,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.628},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.749}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":962560,"data_stored":474760,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63}]},{"osd":2,"last update":"Fri Nov 17 00:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Nov 17 00:06:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60099999999999998,"15min":0.56799999999999995},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.433,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81699999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Nov 17 00:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.629},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93899999999999995,"5min":1.123,"15min":1.123},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.88300000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.55000000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.872}]},{"osd":9,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85499999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.75}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7276,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737876,"statfs":{"total":95995035648,"available":95987585024,"internally_reserved":0,"allocated":954368,"data_stored":471056,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59599999999999997,"15min":0.56200000000000006},"min":{"1min":0.42699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.62}]},{"osd":2,"last update":"Fri Nov 17 00:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.59499999999999997},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.871},{"interface":"front","average":{"1min":0.748,"5min":0.63600000000000001,"15min":0.58999999999999997},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Nov 17 00:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56100000000000005,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95299999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63900000000000001,"15min":0.58199999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0660000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66500000000000004,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.374,"15min":0.374},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.67400000000000004,"15min":0.64900000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.98299999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94399999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.70699999999999996,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.755,"15min":0.74099999999999999},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.054,"5min":1.161,"15min":1.161},"last":0.745}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0209999999999999,"5min":1.377,"15min":1.377},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81899999999999995,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.022}]},{"osd":9,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":1.099},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97299999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.137}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":474794,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51400000000000001,"15min":0.47799999999999998},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.50700000000000001},"min":{"1min":0.42099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77300000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.754},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59699999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Nov 17 00:05:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63400000000000001,"15min":0.61499999999999999},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83299999999999996,"5min":0.875,"15min":0.875},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.51300000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64000000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.626},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.0069999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6636,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738516,"statfs":{"total":95995035648,"available":95988240384,"internally_reserved":0,"allocated":954368,"data_stored":470739,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.44},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.375,"5min":0.255,"15min":0.255},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.88,"15min":0.88},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.63800000000000001,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.66700000000000004,"15min":0.63},"min":{"1min":0.53700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.627},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":1.008},"last":0.747}]},{"osd":4,"last update":"Fri Nov 17 00:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92800000000000005,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Nov 17 00:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.69799999999999995,"15min":0.66400000000000003},"min":{"1min":0.51500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.70699999999999996,"15min":0.67200000000000004},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Nov 17 00:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.131,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71299999999999997,"15min":0.68300000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.755},"min":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2789999999999999,"5min":1.46,"15min":1.46},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.753,"15min":0.71699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.288,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.51800000000000002}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.90000000000000002,"15min":0.92600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.004,"15min":1.034},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2030000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6508,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738644,"statfs":{"total":95995035648,"available":95988371456,"internally_reserved":0,"allocated":954368,"data_stored":470988,"data_compressed":11677,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.29899999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54600000000000004,"15min":0.51600000000000001},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59199999999999997,"15min":0.53900000000000003},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.441,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.56599999999999995},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85399999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Nov 17 00:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.67100000000000004,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.44,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Nov 17 00:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.443,"15min":0.443},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6632,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738520,"statfs":{"total":95995035648,"available":95988244480,"internally_reserved":0,"allocated":1015808,"data_stored":1065612,"data_compressed":47507,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73599999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.437}]},{"osd":1,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.29699999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.372},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83499999999999996,"5min":0.877,"15min":0.877},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 17 00:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.49399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0109999999999999,"5min":1.046,"15min":1.046},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.627},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65300000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":21369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:06:41.822 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-17T00:06:41.823 INFO:tasks.cephadm:Setting up client nodes... 2023-11-17T00:06:41.823 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-17T00:06:42.711 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:42 smithi186 ceph-mon[131487]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:42 smithi186 ceph-mon[131487]: pgmap v308: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2023-11-17T00:06:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:42 smithi186 ceph-mon[131487]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:43.128 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:42 smithi006 ceph-mon[97840]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:43.128 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:42 smithi006 ceph-mon[97840]: pgmap v308: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2023-11-17T00:06:43.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:42 smithi006 ceph-mon[97840]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:42 smithi087 ceph-mon[131182]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:42 smithi087 ceph-mon[131182]: pgmap v308: 129 pgs: 129 active+clean; 598 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2023-11-17T00:06:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:42 smithi087 ceph-mon[131182]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 -- 172.21.15.6:0/3400888095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f175010c8d0 msgr2=0x7f175010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 --2- 172.21.15.6:0/3400888095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f175010c8d0 0x7f175010ecb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f174800b0d0 tx=0x7f174800b3e0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 -- 172.21.15.6:0/3400888095 shutdown_connections 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 --2- 172.21.15.6:0/3400888095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f175010c8d0 0x7f175010ecb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 --2- 172.21.15.6:0/3400888095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1750074b20 0x7f1750072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 --2- 172.21.15.6:0/3400888095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750073550 0x7f17500739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 -- 172.21.15.6:0/3400888095 >> 172.21.15.6:0/3400888095 conn(0x7f17500fb900 msgr2=0x7f17500fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:44.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 -- 172.21.15.6:0/3400888095 shutdown_connections 2023-11-17T00:06:44.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.268+0000 7f1757003700 1 -- 172.21.15.6:0/3400888095 wait complete. 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 Processor -- start 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 -- start start 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f175010c8d0 0x7f17501a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a42a0 con 0x7f1750073550 2023-11-17T00:06:44.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a43e0 con 0x7f1750074b20 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.269+0000 7f1757003700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501a4570 con 0x7f175010c8d0 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f174ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f17555a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f175010c8d0 0x7f17501a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f174ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38198/0 (socket says 172.21.15.6:38198) 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56350/0 (socket says 172.21.15.6:56350) 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f174ffff700 1 -- 172.21.15.6:0/559456273 learned_addr learned my addr 172.21.15.6:0/559456273 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 -- 172.21.15.6:0/559456273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f175010c8d0 msgr2=0x7f17501a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f175010c8d0 0x7f17501a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 -- 172.21.15.6:0/559456273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 msgr2=0x7f175019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 -- 172.21.15.6:0/559456273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748009d00 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f174ffff700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.270+0000 7f1754d9f700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f174400ea20 tx=0x7f174400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.271+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400cbd0 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.271+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f174400ee60 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.271+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744018680 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.271+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1750112290 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.271+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1750112100 con 0x7f1750073550 2023-11-17T00:06:44.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.272+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1750102420 con 0x7f1750073550 2023-11-17T00:06:44.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.274+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1744010430 con 0x7f1750073550 2023-11-17T00:06:44.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.275+0000 7f174dffb700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 0x7f17400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:44.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.275+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1744010e40 con 0x7f1750073550 2023-11-17T00:06:44.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.276+0000 7f174ffff700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 0x7f17400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:44.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.276+0000 7f174ffff700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 0x7f17400a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f173c011550 tx=0x7f173c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:44.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.280+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1744096230 con 0x7f1750073550 2023-11-17T00:06:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.568+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f175002cfe0 con 0x7f1750073550 2023-11-17T00:06:44.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.573+0000 7f174dffb700 1 -- 172.21.15.6:0/559456273 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1744099850 con 0x7f1750073550 2023-11-17T00:06:44.574 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2023-11-17T00:06:44.574 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCUrlZlzL7oIRAAF9hVNbLRQ4IPpcmKjQXr3w== 2023-11-17T00:06:44.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 msgr2=0x7f17400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:44.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 0x7f17400a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f173c011550 tx=0x7f173c009260 comp rx=0 tx=0).stop 2023-11-17T00:06:44.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 msgr2=0x7f175019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:44.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f174400ea20 tx=0x7f174400ed30 comp rx=0 tx=0).stop 2023-11-17T00:06:44.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 shutdown_connections 2023-11-17T00:06:44.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f17400a11e0 0x7f17400a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1750073550 0x7f175019c710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f175010c8d0 0x7f17501a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 --2- 172.21.15.6:0/559456273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1750074b20 0x7f175019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:44.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 >> 172.21.15.6:0/559456273 conn(0x7f17500fb900 msgr2=0x7f17500fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:44.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 shutdown_connections 2023-11-17T00:06:44.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:44.575+0000 7f1757003700 1 -- 172.21.15.6:0/559456273 wait complete. 2023-11-17T00:06:44.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:44 smithi186 ceph-mon[131487]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:44.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:44 smithi006 ceph-mon[97840]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:44.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:44 smithi087 ceph-mon[131182]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:45.016 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:06:45.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-17T00:06:45.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-17T00:06:45.121 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-17T00:06:45.121 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-17T00:06:45.121 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph mgr dump --format=json 2023-11-17T00:06:45.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:45 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-17T00:06:45.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:45 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-17T00:06:45.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:45 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-17T00:06:45.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:45 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-17T00:06:45.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:45 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-17T00:06:45.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:45 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/559456273' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-17T00:06:46.022 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:46.585 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:46 smithi006 ceph-mon[97840]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:46.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:46 smithi186 ceph-mon[131487]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:46.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:46 smithi087 ceph-mon[131182]: pgmap v310: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 -- 172.21.15.6:0/688849725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 msgr2=0x7f1f6c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 --2- 172.21.15.6:0/688849725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c104eb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0099c0 tx=0x7f1f5c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 -- 172.21.15.6:0/688849725 shutdown_connections 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 --2- 172.21.15.6:0/688849725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 --2- 172.21.15.6:0/688849725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c100540 0x7f1f6c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 --2- 172.21.15.6:0/688849725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c104eb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 -- 172.21.15.6:0/688849725 >> 172.21.15.6:0/688849725 conn(0x7f1f6c0fbdc0 msgr2=0x7f1f6c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:47.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 -- 172.21.15.6:0/688849725 shutdown_connections 2023-11-17T00:06:47.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.582+0000 7f1f73f30700 1 -- 172.21.15.6:0/688849725 wait complete. 2023-11-17T00:06:47.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.583+0000 7f1f73f30700 1 Processor -- start 2023-11-17T00:06:47.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 -- start start 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c100540 0x7f1f6c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19b390 con 0x7f1f6c100ee0 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19b500 con 0x7f1f6c1055e0 2023-11-17T00:06:47.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.584+0000 7f1f73f30700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19b6c0 con 0x7f1f6c100540 2023-11-17T00:06:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56356/0 (socket says 172.21.15.6:56356) 2023-11-17T00:06:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 -- 172.21.15.6:0/2264324788 learned_addr learned my addr 172.21.15.6:0/2264324788 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f724cd700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f71ccc700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c100540 0x7f1f6c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 -- 172.21.15.6:0/2264324788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c100540 msgr2=0x7f1f6c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c100540 0x7f1f6c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 -- 172.21.15.6:0/2264324788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 msgr2=0x7f1f6c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 -- 172.21.15.6:0/2264324788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68009520 con 0x7f1f6c100ee0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f724cd700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.585+0000 7f1f714cb700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098e0 tx=0x7f1f5c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.586+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c01d070 con 0x7f1f6c100ee0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.586+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f5c005050 con 0x7f1f6c100ee0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.586+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004970 con 0x7f1f6c100ee0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.586+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0095f0 con 0x7f1f6c100ee0 2023-11-17T00:06:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.586+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c110360 con 0x7f1f6c100ee0 2023-11-17T00:06:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.587+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c1101d0 con 0x7f1f6c100ee0 2023-11-17T00:06:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.589+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1f5c005430 con 0x7f1f6c100ee0 2023-11-17T00:06:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.590+0000 7f1f62ffd700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 0x7f1f580a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.590+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f5c0cdd80 con 0x7f1f6c100ee0 2023-11-17T00:06:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.590+0000 7f1f71ccc700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 0x7f1f580a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.591+0000 7f1f71ccc700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 0x7f1f580a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009df0 tx=0x7f1f68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.592+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1f5c0971b0 con 0x7f1f6c100ee0 2023-11-17T00:06:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.858+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1f6c1104f0 con 0x7f1f6c100ee0 2023-11-17T00:06:47.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.864+0000 7f1f62ffd700 1 -- 172.21.15.6:0/2264324788 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260113 (secure 0 0 0) 0x7f1f5c09a7d0 con 0x7f1f6c100ee0 2023-11-17T00:06:47.871 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:47.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 msgr2=0x7f1f580a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:47.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 0x7f1f580a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009df0 tx=0x7f1f68009260 comp rx=0 tx=0).stop 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 msgr2=0x7f1f6c196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098e0 tx=0x7f1f5c00f690 comp rx=0 tx=0).stop 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 shutdown_connections 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1f580a1260 0x7f1f580a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f6c1055e0 0x7f1f6c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c100540 0x7f1f6c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 --2- 172.21.15.6:0/2264324788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f6c100ee0 0x7f1f6c196910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:47.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 >> 172.21.15.6:0/2264324788 conn(0x7f1f6c0fbdc0 msgr2=0x7f1f6c0fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:47.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 shutdown_connections 2023-11-17T00:06:47.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:47.868+0000 7f1f73f30700 1 -- 172.21.15.6:0/2264324788 wait complete. 2023-11-17T00:06:48.326 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":879660796},{"type":"v1","addr":"172.21.15.6:6801","nonce":879660796}]},"active_addr":"172.21.15.6:6801/879660796","active_change":"2023-11-16T23:59:01.653488+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":4272143857}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1767647821}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2273227531}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3662483346}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":903847005}]}]} 2023-11-17T00:06:48.342 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-17T00:06:48.343 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-17T00:06:48.343 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-17T00:06:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:48 smithi087 ceph-mon[131182]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:48 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2264324788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-17T00:06:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:48 smithi186 ceph-mon[131487]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:48 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2264324788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-17T00:06:49.205 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:48 smithi006 ceph-mon[97840]: pgmap v311: 129 pgs: 129 active+clean; 620 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:48 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2264324788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-17T00:06:50.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 -- 172.21.15.6:0/3208597687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc100540 msgr2=0x7fb0fc100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:50.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 --2- 172.21.15.6:0/3208597687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc100540 0x7fb0fc100910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0099c0 tx=0x7fb0ec009cd0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 -- 172.21.15.6:0/3208597687 shutdown_connections 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 --2- 172.21.15.6:0/3208597687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc100540 0x7fb0fc100910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 --2- 172.21.15.6:0/3208597687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 --2- 172.21.15.6:0/3208597687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100ee0 0x7fb0fc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 -- 172.21.15.6:0/3208597687 >> 172.21.15.6:0/3208597687 conn(0x7fb0fc0fbdc0 msgr2=0x7fb0fc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 -- 172.21.15.6:0/3208597687 shutdown_connections 2023-11-17T00:06:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.767+0000 7fb10125e700 1 -- 172.21.15.6:0/3208597687 wait complete. 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.768+0000 7fb10125e700 1 Processor -- start 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.768+0000 7fb10125e700 1 -- start start 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100540 0x7fb0fc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc100ee0 0x7fb0fc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1a4860 con 0x7fb0fc1055e0 2023-11-17T00:06:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1a49d0 con 0x7fb0fc100ee0 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb10125e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1a4b90 con 0x7fb0fc100540 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb0fb59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb0fb59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39506/0 (socket says 172.21.15.6:39506) 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb0fb59e700 1 -- 172.21.15.6:0/3304971067 learned_addr learned my addr 172.21.15.6:0/3304971067 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb0fad9d700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100540 0x7fb0fc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.769+0000 7fb0fa59c700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc100ee0 0x7fb0fc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 -- 172.21.15.6:0/3304971067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100540 msgr2=0x7fb0fc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100540 0x7fb0fc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 -- 172.21.15.6:0/3304971067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc100ee0 msgr2=0x7fb0fc073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc100ee0 0x7fb0fc073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 -- 172.21.15.6:0/3304971067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0095f0 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0fb59e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4007ee0 tx=0x7fb0f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400dd20 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f400f040 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40048a0 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc110370 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.770+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1101e0 con 0x7fb0fc1055e0 2023-11-17T00:06:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.771+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc110500 con 0x7fb0fc1055e0 2023-11-17T00:06:50.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.773+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb0f4003a00 con 0x7fb0fc1055e0 2023-11-17T00:06:50.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.775+0000 7fb0e3fff700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 0x7fb0e40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:50.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.775+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0f40cc480 con 0x7fb0fc1055e0 2023-11-17T00:06:50.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.775+0000 7fb0fad9d700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 0x7fb0e40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:50.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.776+0000 7fb0fad9d700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 0x7fb0e40a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec000c00 tx=0x7fb0ec019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:50.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.779+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb0f404e730 con 0x7fb0fc1055e0 2023-11-17T00:06:50.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.987+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0fc066ea0 con 0x7fb0fc1055e0 2023-11-17T00:06:50.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.989+0000 7fb0e3fff700 1 -- 172.21.15.6:0/3304971067 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fb0f4095820 con 0x7fb0fc1055e0 2023-11-17T00:06:50.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:50.991 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.991+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 msgr2=0x7fb0e40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.991+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 0x7fb0e40a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec000c00 tx=0x7fb0ec019040 comp rx=0 tx=0).stop 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.991+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 msgr2=0x7fb0fc0738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.991+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4007ee0 tx=0x7fb0f400d3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 shutdown_connections 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb0e40a1260 0x7fb0e40a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0fc1055e0 0x7fb0fc0738b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:50.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0fc100540 0x7fb0fc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:51.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 --2- 172.21.15.6:0/3304971067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0fc100ee0 0x7fb0fc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:51.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 >> 172.21.15.6:0/3304971067 conn(0x7fb0fc0fbdc0 msgr2=0x7fb0fc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:51.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 shutdown_connections 2023-11-17T00:06:51.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:50.992+0000 7fb10125e700 1 -- 172.21.15.6:0/3304971067 wait complete. 2023-11-17T00:06:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:50 smithi087 ceph-mon[131182]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2023-11-17T00:06:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:50 smithi186 ceph-mon[131487]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2023-11-17T00:06:51.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:50 smithi006 ceph-mon[97840]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2023-11-17T00:06:51.606 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-17T00:06:51.606 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-17T00:06:51.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:51 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3304971067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:52.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:51 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3304971067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:52.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:51 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3304971067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:52.472 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:53.034 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:52 smithi006 ceph-mon[97840]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:52 smithi087 ceph-mon[131182]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:52 smithi186 ceph-mon[131487]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 -- 172.21.15.6:0/106473481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac100540 msgr2=0x7f94ac100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/106473481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac100540 0x7f94ac100910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f949c0099c0 tx=0x7f949c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 -- 172.21.15.6:0/106473481 shutdown_connections 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/106473481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac100540 0x7f94ac100910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/106473481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac1055e0 0x7f94ac10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/106473481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100ee0 0x7f94ac104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 -- 172.21.15.6:0/106473481 >> 172.21.15.6:0/106473481 conn(0x7f94ac0fbdc0 msgr2=0x7f94ac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:54.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 -- 172.21.15.6:0/106473481 shutdown_connections 2023-11-17T00:06:54.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.024+0000 7f94b1a0f700 1 -- 172.21.15.6:0/106473481 wait complete. 2023-11-17T00:06:54.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.026+0000 7f94b1a0f700 1 Processor -- start 2023-11-17T00:06:54.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.026+0000 7f94b1a0f700 1 -- start start 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.026+0000 7f94b1a0f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 0x7f94ac198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.026+0000 7f94b1a0f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac100ee0 0x7f94ac198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94b1a0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94b1a0f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19d5c0 con 0x7f94ac1055e0 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94b1a0f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19d730 con 0x7f94ac100ee0 2023-11-17T00:06:54.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94b1a0f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19d8f0 con 0x7f94ac100540 2023-11-17T00:06:54.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:54.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 0x7f94ac198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:54.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39530/0 (socket says 172.21.15.6:39530) 2023-11-17T00:06:54.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 -- 172.21.15.6:0/3859586152 learned_addr learned my addr 172.21.15.6:0/3859586152 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 0x7f94ac198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33566/0 (socket says 172.21.15.6:33566) 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94aa7fc700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac100ee0 0x7f94ac198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 -- 172.21.15.6:0/3859586152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 msgr2=0x7f94ac198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 0x7f94ac198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 -- 172.21.15.6:0/3859586152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac100ee0 msgr2=0x7f94ac198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac100ee0 0x7f94ac198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.027+0000 7f94ab7fe700 1 -- 172.21.15.6:0/3859586152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0095f0 con 0x7f94ac1055e0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94ab7fe700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400dd20 con 0x7f94ac1055e0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94a400f040 con 0x7f94ac1055e0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40048a0 con 0x7f94ac1055e0 2023-11-17T00:06:54.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac110370 con 0x7f94ac1055e0 2023-11-17T00:06:54.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.028+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac1101e0 con 0x7f94ac1055e0 2023-11-17T00:06:54.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.029+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac110500 con 0x7f94ac1055e0 2023-11-17T00:06:54.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.031+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f94a4003a00 con 0x7f94ac1055e0 2023-11-17T00:06:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.032+0000 7f94b0a0d700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 0x7f94940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:06:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.033+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f94a40cc500 con 0x7f94ac1055e0 2023-11-17T00:06:54.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.033+0000 7f94aaffd700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 0x7f94940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:06:54.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.033+0000 7f94aaffd700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 0x7f94940a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f949c009990 tx=0x7f949c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:06:54.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.037+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f94a404e730 con 0x7f94ac1055e0 2023-11-17T00:06:54.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.246+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94ac066ea0 con 0x7f94ac1055e0 2023-11-17T00:06:54.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.248+0000 7f94b0a0d700 1 -- 172.21.15.6:0/3859586152 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f94a4095070 con 0x7f94ac1055e0 2023-11-17T00:06:54.249 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:06:54.249 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":88,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:06:17.601344+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 msgr2=0x7f94940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 0x7f94940a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f949c009990 tx=0x7f949c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 msgr2=0x7f94ac19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 shutdown_connections 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f94940a11e0 0x7f94940a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac1055e0 0x7f94ac19cfb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ac100540 0x7f94ac198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 --2- 172.21.15.6:0/3859586152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94ac100ee0 0x7f94ac198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:06:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.250+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 >> 172.21.15.6:0/3859586152 conn(0x7f94ac0fbdc0 msgr2=0x7f94ac0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:06:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.251+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 shutdown_connections 2023-11-17T00:06:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:06:54.251+0000 7f94b1a0f700 1 -- 172.21.15.6:0/3859586152 wait complete. 2023-11-17T00:06:54.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:54 smithi186 ceph-mon[131487]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2023-11-17T00:06:56.007 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:54 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3859586152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:56.008 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:54 smithi087 ceph-mon[131182]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2023-11-17T00:06:56.008 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:54 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3859586152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:56.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:54 smithi006 ceph-mon[97840]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2023-11-17T00:06:56.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:54 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3859586152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:06:56.014 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-11-17T00:06:56.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-11-17T00:06:56.018 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-11-17T00:06:56.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-11-17T00:06:56.021 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-11-17T00:06:56.023 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-11-17T00:06:56.023 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-11-17T00:06:56.153 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-11-17T00:06:56.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-11-17T00:06:56.155 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-11-17T00:06:56.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-11-17T00:06:56.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-11-17T00:06:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:56 smithi087 ceph-mon[131182]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:06:57.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:56 smithi186 ceph-mon[131487]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:06:57.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:56 smithi006 ceph-mon[97840]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:06:58.017 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.052 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.467 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.844 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.890 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:06:58 smithi006 ceph-mon[97840]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:06:58.896 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:58.972 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:06:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:06:58 smithi087 ceph-mon[131182]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:06:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:06:58 smithi186 ceph-mon[131487]: pgmap v316: 129 pgs: 129 active+clean; 636 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:07:00.916 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:00 smithi006 ceph-mon[97840]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:07:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:00 smithi087 ceph-mon[131182]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:07:01.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:00 smithi186 ceph-mon[131487]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 -- 172.21.15.6:0/500766702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101540 msgr2=0x7fd728101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 --2- 172.21.15.6:0/500766702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101540 0x7fd728101910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd7180098f0 tx=0x7fd718009c00 comp rx=0 tx=0).stop 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 -- 172.21.15.6:0/500766702 shutdown_connections 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 --2- 172.21.15.6:0/500766702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101540 0x7fd728101910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 --2- 172.21.15.6:0/500766702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 0x7fd72810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 --2- 172.21.15.6:0/500766702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101e50 0x7fd72810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 -- 172.21.15.6:0/500766702 >> 172.21.15.6:0/500766702 conn(0x7fd7280770a0 msgr2=0x7fd7280774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 -- 172.21.15.6:0/500766702 shutdown_connections 2023-11-17T00:07:01.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.331+0000 7fd72e643700 1 -- 172.21.15.6:0/500766702 wait complete. 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.332+0000 7fd72e643700 1 Processor -- start 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.332+0000 7fd72e643700 1 -- start start 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 0x7fd7281a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101e50 0x7fd7281a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 0x7fd728109010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7281097b0 con 0x7fd728101e50 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728109920 con 0x7fd728101540 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72e643700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728109ae0 con 0x7fd72810a890 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101e50 0x7fd7281a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 0x7fd7281a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:01.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 0x7fd7281a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59522/0 (socket says 172.21.15.6:59522) 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 -- 172.21.15.6:0/3646520761 learned_addr learned my addr 172.21.15.6:0/3646520761 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72cbe0700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 0x7fd728109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 -- 172.21.15.6:0/3646520761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 msgr2=0x7fd728109010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 0x7fd728109010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 -- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101e50 msgr2=0x7fd7281a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101e50 0x7fd7281a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd727fff700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0095f0 con 0x7fd728101540 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd72cbe0700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd72810a890 0x7fd728109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.333+0000 7fd7277fe700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd728101e50 0x7fd7281a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd727fff700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 0x7fd7281a1740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd718009810 tx=0x7fd71800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71801c070 con 0x7fd728101540 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd718009520 con 0x7fd728101540 2023-11-17T00:07:01.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71800bb20 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718004930 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.334+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd728110360 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.335+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7280fd790 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.337+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd718020070 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7257fa700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd7100a11e0 0x7fd7100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7180ce0b0 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7277fe700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd7100a11e0 0x7fd7100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7257fa700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] conn(0x7fd7100abd50 0x7fd7100ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7100ae850 con 0x7fd7100abd50 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7180ce420 con 0x7fd728101540 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd72cbe0700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] conn(0x7fd7100abd50 0x7fd7100ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:01.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.339+0000 7fd7277fe700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd7100a11e0 0x7fd7100a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd72819e1e0 tx=0x7fd71c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.340+0000 7fd72cbe0700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] conn(0x7fd7100abd50 0x7fd7100ae180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.341+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== osd.3 v2:172.21.15.6:6810/1124657350 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd7100ae850 con 0x7fd7100abd50 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.357+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 --> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd728109ea0 con 0x7fd7100abd50 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd7257fa700 1 -- 172.21.15.6:0/3646520761 <== osd.3 v2:172.21.15.6:6810/1124657350 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd72000f040 con 0x7fd7100abd50 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] conn(0x7fd7100abd50 msgr2=0x7fd7100ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] conn(0x7fd7100abd50 0x7fd7100ae180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd7100a11e0 msgr2=0x7fd7100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd7100a11e0 0x7fd7100a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd72819e1e0 tx=0x7fd71c009310 comp rx=0 tx=0).stop 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 msgr2=0x7fd7281a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 --2- 172.21.15.6:0/3646520761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728101540 0x7fd7281a1740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd718009810 tx=0x7fd71800fa00 comp rx=0 tx=0).stop 2023-11-17T00:07:01.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72cbe0700 1 -- 172.21.15.6:0/3646520761 reap_dead start 2023-11-17T00:07:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 shutdown_connections 2023-11-17T00:07:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 >> 172.21.15.6:0/3646520761 conn(0x7fd7280770a0 msgr2=0x7fd7280fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 shutdown_connections 2023-11-17T00:07:01.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:01.358+0000 7fd72e643700 1 -- 172.21.15.6:0/3646520761 wait complete. 2023-11-17T00:07:01.812 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:01.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-17T00:07:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:01 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2023-11-17T00:07:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-11-17T00:07:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-17T00:07:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:01 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2023-11-17T00:07:02.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-11-17T00:07:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-17T00:07:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:01 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-17T00:07:02.743 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-17T00:07:03.081 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-17T00:07:03.082 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:02 smithi006 ceph-mon[97840]: osdmap e89: 12 total, 12 up, 12 in 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-17T00:07:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:02 smithi087 ceph-mon[131182]: osdmap e89: 12 total, 12 up, 12 in 2023-11-17T00:07:03.107 INFO:teuthology.orchestra.run.smithi006.stdout:107374182462 2023-11-17T00:07:03.107 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-17T00:07:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-17T00:07:03.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:02 smithi186 ceph-mon[131487]: osdmap e89: 12 total, 12 up, 12 in 2023-11-17T00:07:03.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 -- 172.21.15.6:0/255253964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100540 msgr2=0x7f4dfc100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/255253964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100540 0x7f4dfc100910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4df80099c0 tx=0x7f4df8009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 -- 172.21.15.6:0/255253964 shutdown_connections 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/255253964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dfc1055e0 0x7f4dfc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/255253964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100540 0x7f4dfc100910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/255253964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc100ee0 0x7f4dfc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 -- 172.21.15.6:0/255253964 >> 172.21.15.6:0/255253964 conn(0x7f4dfc0fbdc0 msgr2=0x7f4dfc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.625+0000 7f4e03ff2700 1 -- 172.21.15.6:0/255253964 shutdown_connections 2023-11-17T00:07:03.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.626+0000 7f4e03ff2700 1 -- 172.21.15.6:0/255253964 wait complete. 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.626+0000 7f4e03ff2700 1 Processor -- start 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 -- start start 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dfc100540 0x7f4dfc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100ee0 0x7f4dfc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc1055e0 0x7f4dfc0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1a47f0 con 0x7f4dfc100540 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1a4960 con 0x7f4dfc1055e0 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e03ff2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc1a4b20 con 0x7f4dfc100ee0 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e0158d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100ee0 0x7f4dfc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:03.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e0158d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100ee0 0x7f4dfc073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48510/0 (socket says 172.21.15.6:48510) 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e0158d700 1 -- 172.21.15.6:0/4100258202 learned_addr learned my addr 172.21.15.6:0/4100258202 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e01d8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dfc100540 0x7f4dfc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.627+0000 7f4e0258f700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc1055e0 0x7f4dfc0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 -- 172.21.15.6:0/4100258202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100ee0 msgr2=0x7f4dfc073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dfc100ee0 0x7f4dfc073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 -- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dfc100540 msgr2=0x7f4dfc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dfc100540 0x7f4dfc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009520 con 0x7f4dfc1055e0 2023-11-17T00:07:03.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e0258f700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc1055e0 0x7f4dfc0738b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4df400efb0 tx=0x7f4df400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4003a00 con 0x7f4dfc1055e0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df400f040 con 0x7f4dfc1055e0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4009c40 con 0x7f4dfc1055e0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.628+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4004540 con 0x7f4dfc1055e0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.629+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df80095f0 con 0x7f4dfc1055e0 2023-11-17T00:07:03.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.629+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4dfc110450 con 0x7f4dfc1055e0 2023-11-17T00:07:03.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.632+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4df4003bf0 con 0x7f4dfc1055e0 2023-11-17T00:07:03.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.633+0000 7f4df2ffd700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4de80a11e0 0x7f4de80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:03.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4df40cc990 con 0x7f4dfc1055e0 2023-11-17T00:07:03.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4e01d8e700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4de80a11e0 0x7f4de80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4df2ffd700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] conn(0x7f4de80ac1f0 0x7f4de80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4de80aed50 con 0x7f4de80ac1f0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df40ccd00 con 0x7f4dfc1055e0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4e0158d700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] conn(0x7f4de80ac1f0 0x7f4de80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.634+0000 7f4e01d8e700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4de80a11e0 0x7f4de80a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009fd0 tx=0x7f4df8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.635+0000 7f4e0158d700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] conn(0x7f4de80ac1f0 0x7f4de80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:03.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.636+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== osd.6 v2:172.21.15.6:6818/1113025990 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4de80aed50 con 0x7f4de80ac1f0 2023-11-17T00:07:03.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.652+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 --> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4dfc066ea0 con 0x7f4de80ac1f0 2023-11-17T00:07:03.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4df2ffd700 1 -- 172.21.15.6:0/4100258202 <== osd.6 v2:172.21.15.6:6818/1113025990 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4dfc066ea0 con 0x7f4de80ac1f0 2023-11-17T00:07:03.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] conn(0x7f4de80ac1f0 msgr2=0x7f4de80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] conn(0x7f4de80ac1f0 0x7f4de80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4de80a11e0 msgr2=0x7f4de80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4de80a11e0 0x7f4de80a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009fd0 tx=0x7f4df8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc1055e0 msgr2=0x7f4dfc0738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e03ff2700 1 --2- 172.21.15.6:0/4100258202 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4dfc1055e0 0x7f4dfc0738b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4df400efb0 tx=0x7f4df400c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.653+0000 7f4e0258f700 1 -- 172.21.15.6:0/4100258202 reap_dead start 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.654+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 shutdown_connections 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.654+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 >> 172.21.15.6:0/4100258202 conn(0x7f4dfc0fbdc0 msgr2=0x7f4dfc0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.654+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 shutdown_connections 2023-11-17T00:07:03.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.654+0000 7f4e03ff2700 1 -- 172.21.15.6:0/4100258202 wait complete. 2023-11-17T00:07:03.955 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:03 smithi006 ceph-mon[97840]: osdmap e90: 12 total, 12 up, 12 in 2023-11-17T00:07:03.967 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:04.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:03 smithi087 ceph-mon[131182]: osdmap e90: 12 total, 12 up, 12 in 2023-11-17T00:07:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:03 smithi186 ceph-mon[131487]: osdmap e90: 12 total, 12 up, 12 in 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3736227926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac0100ee0 msgr2=0x7f3ac0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3736227926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac0100ee0 0x7f3ac0104eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40099c0 tx=0x7f3ab4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3736227926 shutdown_connections 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3736227926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ac01055e0 0x7f3ac010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3736227926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac0100540 0x7f3ac0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3736227926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac0100ee0 0x7f3ac0104eb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3736227926 >> 172.21.15.6:0/3736227926 conn(0x7f3ac00fbdc0 msgr2=0x7f3ac00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:04.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3736227926 shutdown_connections 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.912+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3736227926 wait complete. 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.914+0000 7f3ac4ec3700 1 Processor -- start 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.914+0000 7f3ac4ec3700 1 -- start start 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac0100540 0x7f3ac0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ac0100ee0 0x7f3ac0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 0x7f3ac0073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a4810 con 0x7f3ac01055e0 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a4980 con 0x7f3ac0100ee0 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3ac4ec3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01a4b40 con 0x7f3ac0100540 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 0x7f3ac0073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 0x7f3ac0073870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56394/0 (socket says 172.21.15.6:56394) 2023-11-17T00:07:04.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 -- 172.21.15.6:0/3189509140 learned_addr learned my addr 172.21.15.6:0/3189509140 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abdd9b700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ac0100ee0 0x7f3ac0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abe59c700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac0100540 0x7f3ac0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 -- 172.21.15.6:0/3189509140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac0100540 msgr2=0x7f3ac0072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ac0100540 0x7f3ac0072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 -- 172.21.15.6:0/3189509140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ac0100ee0 msgr2=0x7f3ac0073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ac0100ee0 0x7f3ac0073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.915+0000 7f3abed9d700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40095f0 con 0x7f3ac01055e0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3abed9d700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 0x7f3ac0073870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8007ee0 tx=0x7f3ab800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800dd20 con 0x7f3ac01055e0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab800f040 con 0x7f3ac01055e0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac0110360 con 0x7f3ac01055e0 2023-11-17T00:07:04.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80048a0 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.916+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac01101d0 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.917+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ac01104f0 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.919+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3ab8003a00 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.920+0000 7f3aaf7fe700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3aa80a11e0 0x7f3aa80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ab80cd120 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3abe59c700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3aa80a11e0 0x7f3aa80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3aaf7fe700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] conn(0x7f3aa80ac1f0 0x7f3aa80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3aa80aed50 con 0x7f3aa80ac1f0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab80cd490 con 0x7f3ac01055e0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3abdd9b700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] conn(0x7f3aa80ac1f0 0x7f3aa80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:04.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.921+0000 7f3abe59c700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3aa80a11e0 0x7f3aa80a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009c60 tx=0x7f3ab0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.922+0000 7f3abdd9b700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] conn(0x7f3aa80ac1f0 0x7f3aa80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.923+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== osd.0 v2:172.21.15.6:6802/3863583460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3aa80aed50 con 0x7f3aa80ac1f0 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.940+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 --> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ac00fd1a0 con 0x7f3aa80ac1f0 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.941+0000 7f3aaf7fe700 1 -- 172.21.15.6:0/3189509140 <== osd.0 v2:172.21.15.6:6802/3863583460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ac00fd1a0 con 0x7f3aa80ac1f0 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] conn(0x7f3aa80ac1f0 msgr2=0x7f3aa80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] conn(0x7f3aa80ac1f0 0x7f3aa80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3aa80a11e0 msgr2=0x7f3aa80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3aa80a11e0 0x7f3aa80a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009c60 tx=0x7f3ab0009260 comp rx=0 tx=0).stop 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 msgr2=0x7f3ac0073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 --2- 172.21.15.6:0/3189509140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ac01055e0 0x7f3ac0073870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8007ee0 tx=0x7f3ab800d3b0 comp rx=0 tx=0).stop 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3abed9d700 1 -- 172.21.15.6:0/3189509140 reap_dead start 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 shutdown_connections 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 >> 172.21.15.6:0/3189509140 conn(0x7f3ac00fbdc0 msgr2=0x7f3ac00fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 shutdown_connections 2023-11-17T00:07:04.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:03.942+0000 7f3ac4ec3700 1 -- 172.21.15.6:0/3189509140 wait complete. 2023-11-17T00:07:04.222 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:04.224 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:04 smithi006 ceph-mon[97840]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:04 smithi087 ceph-mon[131182]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:04 smithi186 ceph-mon[131487]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 -- 172.21.15.6:0/2413696513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 msgr2=0x7f920c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 --2- 172.21.15.6:0/2413696513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 0x7f920c108a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92080098f0 tx=0x7f9208009c00 comp rx=0 tx=0).stop 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 -- 172.21.15.6:0/2413696513 shutdown_connections 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 --2- 172.21.15.6:0/2413696513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c102f50 0x7f920c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 --2- 172.21.15.6:0/2413696513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 0x7f920c108a80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 --2- 172.21.15.6:0/2413696513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c1025c0 0x7f920c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 -- 172.21.15.6:0/2413696513 >> 172.21.15.6:0/2413696513 conn(0x7f920c0fe100 msgr2=0x7f920c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 -- 172.21.15.6:0/2413696513 shutdown_connections 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.427+0000 7f92141fa700 1 -- 172.21.15.6:0/2413696513 wait complete. 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.428+0000 7f92141fa700 1 Processor -- start 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.428+0000 7f92141fa700 1 -- start start 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 0x7f920c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c102f50 0x7f920c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 0x7f920c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d590 con 0x7f920c1025c0 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d700 con 0x7f920c102f50 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f92141fa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d8c0 con 0x7f920c1086b0 2023-11-17T00:07:05.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 0x7f920c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 0x7f920c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56436/0 (socket says 172.21.15.6:56436) 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 -- 172.21.15.6:0/208640826 learned_addr learned my addr 172.21.15.6:0/208640826 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211795700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c102f50 0x7f920c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 -- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 msgr2=0x7f920c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f920c1086b0 0x7f920c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 -- 172.21.15.6:0/208640826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c102f50 msgr2=0x7f920c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f920c102f50 0x7f920c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.429+0000 7f9211f96700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204009c40 con 0x7f920c1025c0 2023-11-17T00:07:05.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f9211f96700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 0x7f920c1985d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9208009810 tx=0x7f920800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920801c070 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9208005040 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92080049a0 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9208009520 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.430+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c112600 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.431+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f920c100280 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.433+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9208020070 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.434+0000 7f9202ffd700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91f80a1260 0x7f91f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.434+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f92080ce5a0 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.435+0000 7f9211795700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91f80a1260 0x7f91f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.435+0000 7f9202ffd700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] conn(0x7f91f80ac270 0x7f91f80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.435+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91f80aedd0 con 0x7f91f80ac270 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.435+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92080ce910 con 0x7f920c1025c0 2023-11-17T00:07:05.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.435+0000 7f9212797700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] conn(0x7f91f80ac270 0x7f91f80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.436+0000 7f9211795700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91f80a1260 0x7f91f80a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f920c199bf0 tx=0x7f91fc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.436+0000 7f9212797700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] conn(0x7f91f80ac270 0x7f91f80ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.437+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== osd.5 v2:172.21.15.186:6808/2177950803 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f91f80aedd0 con 0x7f91f80ac270 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.456+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 --> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f920c112470 con 0x7f91f80ac270 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f9202ffd700 1 -- 172.21.15.6:0/208640826 <== osd.5 v2:172.21.15.186:6808/2177950803 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f920c112470 con 0x7f91f80ac270 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] conn(0x7f91f80ac270 msgr2=0x7f91f80ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] conn(0x7f91f80ac270 0x7f91f80ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91f80a1260 msgr2=0x7f91f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f91f80a1260 0x7f91f80a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f920c199bf0 tx=0x7f91fc009310 comp rx=0 tx=0).stop 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 msgr2=0x7f920c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 --2- 172.21.15.6:0/208640826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f920c1025c0 0x7f920c1985d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9208009810 tx=0x7f920800fa00 comp rx=0 tx=0).stop 2023-11-17T00:07:05.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f9212797700 1 -- 172.21.15.6:0/208640826 reap_dead start 2023-11-17T00:07:05.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 shutdown_connections 2023-11-17T00:07:05.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.457+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 >> 172.21.15.6:0/208640826 conn(0x7f920c0fe100 msgr2=0x7f920c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:05.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.458+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 shutdown_connections 2023-11-17T00:07:05.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.458+0000 7f92141fa700 1 -- 172.21.15.6:0/208640826 wait complete. 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 -- 172.21.15.6:0/3293381236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100ee0 msgr2=0x7f93c8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 --2- 172.21.15.6:0/3293381236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100ee0 0x7f93c8104eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f93b80099c0 tx=0x7f93b8009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 -- 172.21.15.6:0/3293381236 shutdown_connections 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 --2- 172.21.15.6:0/3293381236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c81055e0 0x7f93c810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 --2- 172.21.15.6:0/3293381236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100540 0x7f93c8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 --2- 172.21.15.6:0/3293381236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100ee0 0x7f93c8104eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 -- 172.21.15.6:0/3293381236 >> 172.21.15.6:0/3293381236 conn(0x7f93c80fbdc0 msgr2=0x7f93c80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 -- 172.21.15.6:0/3293381236 shutdown_connections 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.196+0000 7f93d04d9700 1 -- 172.21.15.6:0/3293381236 wait complete. 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.197+0000 7f93d04d9700 1 Processor -- start 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.197+0000 7f93d04d9700 1 -- start start 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 0x7f93c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100ee0 0x7f93c8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c81055e0 0x7f93c819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c819d5b0 con 0x7f93c8100540 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c819d720 con 0x7f93c81055e0 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93d04d9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c819d8e0 con 0x7f93c8100ee0 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93ce275700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 0x7f93c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93ce275700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 0x7f93c81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56418/0 (socket says 172.21.15.6:56418) 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93ce275700 1 -- 172.21.15.6:0/3266586643 learned_addr learned my addr 172.21.15.6:0/3266586643 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93cda74700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100ee0 0x7f93c8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.198+0000 7f93cea76700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c81055e0 0x7f93c819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 -- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c81055e0 msgr2=0x7f93c819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c81055e0 0x7f93c819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 -- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 msgr2=0x7f93c81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 0x7f93c81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80095f0 con 0x7f93c8100ee0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93ce275700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93c8100540 0x7f93c81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93cda74700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100ee0 0x7f93c8198b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93b80058e0 tx=0x7f93b800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.199+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b801d070 con 0x7f93c8100ee0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.200+0000 7f93d04d9700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c8110360 con 0x7f93c8100ee0 2023-11-17T00:07:05.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.200+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b8005020 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.200+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b80049e0 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.200+0000 7f93d04d9700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c81101d0 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.200+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93b0000fc0 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.203+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f93b8005190 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.204+0000 7f93bf7fe700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f93b40a1260 0x7f93b40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.204+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93b80cdc10 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93ce275700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f93b40a1260 0x7f93b40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93bf7fe700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] conn(0x7f93b40ac270 0x7f93b40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93b40aedd0 con 0x7f93b40ac270 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b80cdf80 con 0x7f93c8100ee0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93cea76700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] conn(0x7f93b40ac270 0x7f93b40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:05.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.205+0000 7f93ce275700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f93b40a1260 0x7f93b40a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93c4011590 tx=0x7f93c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.206+0000 7f93cea76700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] conn(0x7f93b40ac270 0x7f93b40ae700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.206+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== osd.7 v2:172.21.15.87:6816/1542327799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f93b40aedd0 con 0x7f93b40ac270 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.225+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 --> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93b0002d40 con 0x7f93b40ac270 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bf7fe700 1 -- 172.21.15.6:0/3266586643 <== osd.7 v2:172.21.15.87:6816/1542327799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f93b0002d40 con 0x7f93b40ac270 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] conn(0x7f93b40ac270 msgr2=0x7f93b40ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] conn(0x7f93b40ac270 0x7f93b40ae700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:05.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f93b40a1260 msgr2=0x7f93b40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f93b40a1260 0x7f93b40a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93c4011590 tx=0x7f93c4009260 comp rx=0 tx=0).stop 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100ee0 msgr2=0x7f93c8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 --2- 172.21.15.6:0/3266586643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93c8100ee0 0x7f93c8198b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93b80058e0 tx=0x7f93b800f690 comp rx=0 tx=0).stop 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93cea76700 1 -- 172.21.15.6:0/3266586643 reap_dead start 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 shutdown_connections 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 >> 172.21.15.6:0/3266586643 conn(0x7f93c80fbdc0 msgr2=0x7f93c80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.226+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 shutdown_connections 2023-11-17T00:07:05.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:05.227+0000 7f93bd7fa700 1 -- 172.21.15.6:0/3266586643 wait complete. 2023-11-17T00:07:06.005 INFO:teuthology.orchestra.run.smithi006.stdout:197568495665 2023-11-17T00:07:06.006 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-11-17T00:07:06.186 INFO:teuthology.orchestra.run.smithi006.stdout:34359738446 2023-11-17T00:07:06.187 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-11-17T00:07:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 -- 172.21.15.6:0/3193140026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 msgr2=0x7fa6a0104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/3193140026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a0104120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6900098f0 tx=0x7fa690009c00 comp rx=0 tx=0).stop 2023-11-17T00:07:06.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 -- 172.21.15.6:0/3193140026 shutdown_connections 2023-11-17T00:07:06.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/3193140026 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a0104660 0x7fa6a0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/3193140026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a00ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/3193140026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a0104120 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.188+0000 7fa6a78c1700 1 -- 172.21.15.6:0/3193140026 >> 172.21.15.6:0/3193140026 conn(0x7fa6a00770a0 msgr2=0x7fa6a00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.189+0000 7fa6a78c1700 1 -- 172.21.15.6:0/3193140026 shutdown_connections 2023-11-17T00:07:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.189+0000 7fa6a78c1700 1 -- 172.21.15.6:0/3193140026 wait complete. 2023-11-17T00:07:06.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 Processor -- start 2023-11-17T00:07:06.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 -- start start 2023-11-17T00:07:06.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a01a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:06.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a01a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:06.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a0104660 0x7fa6a019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:06.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d040 con 0x7fa6a0104660 2023-11-17T00:07:06.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d1b0 con 0x7fa6a00ffc40 2023-11-17T00:07:06.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.190+0000 7fa6a78c1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d370 con 0x7fa6a00ff330 2023-11-17T00:07:06.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a01a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:06.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a01a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:48574/0 (socket says 172.21.15.6:48574) 2023-11-17T00:07:06.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 -- 172.21.15.6:0/710245083 learned_addr learned my addr 172.21.15.6:0/710245083 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:06.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a4e5c700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a01a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:06.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a5e5e700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a0104660 0x7fa6a019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:06.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 -- 172.21.15.6:0/710245083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 msgr2=0x7fa6a01a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a01a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 -- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a0104660 msgr2=0x7fa6a019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6a0104660 0x7fa6a019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690009520 con 0x7fa6a00ff330 2023-11-17T00:07:06.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a4e5c700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6a00ffc40 0x7fa6a01a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:06.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.191+0000 7fa6a565d700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a01a28a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9a0 tx=0x7fa69c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:06.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.192+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c009790 con 0x7fa6a00ff330 2023-11-17T00:07:06.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.192+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a0116970 con 0x7fa6a00ff330 2023-11-17T00:07:06.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.192+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69c010430 con 0x7fa6a00ff330 2023-11-17T00:07:06.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.192+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01167e0 con 0x7fa6a00ff330 2023-11-17T00:07:06.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.192+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c018680 con 0x7fa6a00ff330 2023-11-17T00:07:06.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.193+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa684000ff0 con 0x7fa6a00ff330 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.195+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa69c00ca10 con 0x7fa6a00ff330 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.196+0000 7fa6967fc700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa68c0a1260 0x7fa68c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.196+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa69c014070 con 0x7fa6a00ff330 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6a4e5c700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa68c0a1260 0x7fa68c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6967fc700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] conn(0x7fa68c0ac270 0x7fa68c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa68c0aedd0 con 0x7fa68c0ac270 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69c0cdb80 con 0x7fa6a00ff330 2023-11-17T00:07:06.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6a5e5e700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] conn(0x7fa68c0ac270 0x7fa68c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:06.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.197+0000 7fa6a4e5c700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa68c0a1260 0x7fa68c0a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa690005e90 tx=0x7fa690019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:06.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.198+0000 7fa6a5e5e700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] conn(0x7fa68c0ac270 0x7fa68c0ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:06.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.199+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== osd.8 v2:172.21.15.186:6816/4126523122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa68c0aedd0 con 0x7fa68c0ac270 2023-11-17T00:07:06.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.219+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 --> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa684002d70 con 0x7fa68c0ac270 2023-11-17T00:07:06.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6967fc700 1 -- 172.21.15.6:0/710245083 <== osd.8 v2:172.21.15.186:6816/4126523122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa684002d70 con 0x7fa68c0ac270 2023-11-17T00:07:06.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] conn(0x7fa68c0ac270 msgr2=0x7fa68c0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] conn(0x7fa68c0ac270 0x7fa68c0ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa68c0a1260 msgr2=0x7fa68c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa68c0a1260 0x7fa68c0a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa690005e90 tx=0x7fa690019040 comp rx=0 tx=0).stop 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 msgr2=0x7fa6a01a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a78c1700 1 --2- 172.21.15.6:0/710245083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff330 0x7fa6a01a28a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9a0 tx=0x7fa69c00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.220+0000 7fa6a5e5e700 1 -- 172.21.15.6:0/710245083 reap_dead start 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.221+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 shutdown_connections 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.221+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 >> 172.21.15.6:0/710245083 conn(0x7fa6a00770a0 msgr2=0x7fa6a0102210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.221+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 shutdown_connections 2023-11-17T00:07:06.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:06.221+0000 7fa6a78c1700 1 -- 172.21.15.6:0/710245083 wait complete. 2023-11-17T00:07:06.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:06 smithi006 ceph-mon[97840]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 -- 172.21.15.6:0/1600831967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 msgr2=0x7f9c2c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 --2- 172.21.15.6:0/1600831967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c100910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009990 tx=0x7f9c1c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 -- 172.21.15.6:0/1600831967 shutdown_connections 2023-11-17T00:07:07.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 --2- 172.21.15.6:0/1600831967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 0x7f9c2c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 --2- 172.21.15.6:0/1600831967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c100910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 --2- 172.21.15.6:0/1600831967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c2c100ee0 0x7f9c2c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.033+0000 7f9c3132e700 1 -- 172.21.15.6:0/1600831967 >> 172.21.15.6:0/1600831967 conn(0x7f9c2c0fbdc0 msgr2=0x7f9c2c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.034+0000 7f9c3132e700 1 -- 172.21.15.6:0/1600831967 shutdown_connections 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.034+0000 7f9c3132e700 1 -- 172.21.15.6:0/1600831967 wait complete. 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.035+0000 7f9c3132e700 1 Processor -- start 2023-11-17T00:07:07.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.035+0000 7f9c3132e700 1 -- start start 2023-11-17T00:07:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c2c100ee0 0x7f9c2c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 0x7f9c2c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19b3a0 con 0x7f9c2c1055e0 2023-11-17T00:07:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19b510 con 0x7f9c2c100540 2023-11-17T00:07:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c3132e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c19b6d0 con 0x7f9c2c100ee0 2023-11-17T00:07:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c2affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c2affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59628/0 (socket says 172.21.15.6:59628) 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c2affd700 1 -- 172.21.15.6:0/2193390692 learned_addr learned my addr 172.21.15.6:0/2193390692 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c2b7fe700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 0x7f9c2c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.036+0000 7f9c2a7fc700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c2c100ee0 0x7f9c2c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 -- 172.21.15.6:0/2193390692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c2c100ee0 msgr2=0x7f9c2c196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c2c100ee0 0x7f9c2c196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 -- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 msgr2=0x7f9c2c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c2c100540 0x7f9c2c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0095f0 con 0x7f9c2c1055e0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c2b7fe700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 0x7f9c2c19ad90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400efc0 tx=0x7f9c2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24018070 con 0x7f9c2c1055e0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.037+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2400f040 con 0x7f9c2c1055e0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.038+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c110370 con 0x7f9c2c1055e0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.038+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400cd40 con 0x7f9c2c1055e0 2023-11-17T00:07:07.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.038+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1101e0 con 0x7f9c2c1055e0 2023-11-17T00:07:07.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.039+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c2c0fd2c0 con 0x7f9c2c1055e0 2023-11-17T00:07:07.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.041+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9c24007500 con 0x7f9c2c1055e0 2023-11-17T00:07:07.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c13fff700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c140a1260 0x7f9c140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c240cd250 con 0x7f9c2c1055e0 2023-11-17T00:07:07.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c13fff700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] conn(0x7f9c140ac270 0x7f9c140ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c140aedd0 con 0x7f9c140ac270 2023-11-17T00:07:07.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c240cd5c0 con 0x7f9c2c1055e0 2023-11-17T00:07:07.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.043+0000 7f9c2affd700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c140a1260 0x7f9c140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.044+0000 7f9c2a7fc700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] conn(0x7f9c140ac270 0x7f9c140ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.044+0000 7f9c2a7fc700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] conn(0x7f9c140ac270 0x7f9c140ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.044+0000 7f9c2affd700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c140a1260 0x7f9c140a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c006010 tx=0x7f9c1c005ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.045+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== osd.10 v2:172.21.15.87:6824/1132375941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9c140aedd0 con 0x7f9c140ac270 2023-11-17T00:07:07.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.062+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 --> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c2c110500 con 0x7f9c140ac270 2023-11-17T00:07:07.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c13fff700 1 -- 172.21.15.6:0/2193390692 <== osd.10 v2:172.21.15.87:6824/1132375941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c2c110500 con 0x7f9c140ac270 2023-11-17T00:07:07.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] conn(0x7f9c140ac270 msgr2=0x7f9c140ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] conn(0x7f9c140ac270 0x7f9c140ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c140a1260 msgr2=0x7f9c140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c140a1260 0x7f9c140a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c006010 tx=0x7f9c1c005ab0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 msgr2=0x7f9c2c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c3132e700 1 --2- 172.21.15.6:0/2193390692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c2c1055e0 0x7f9c2c19ad90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400efc0 tx=0x7f9c2400c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.063+0000 7f9c2b7fe700 1 -- 172.21.15.6:0/2193390692 reap_dead start 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.064+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 shutdown_connections 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.064+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 >> 172.21.15.6:0/2193390692 conn(0x7f9c2c0fbdc0 msgr2=0x7f9c2c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.064+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 shutdown_connections 2023-11-17T00:07:07.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.064+0000 7f9c3132e700 1 -- 172.21.15.6:0/2193390692 wait complete. 2023-11-17T00:07:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:06 smithi087 ceph-mon[131182]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:06 smithi186 ceph-mon[131487]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 -- 172.21.15.6:0/2150171153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8094100ee0 msgr2=0x7f8094104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 --2- 172.21.15.6:0/2150171153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8094100ee0 0x7f8094104eb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80800099c0 tx=0x7f8080009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 -- 172.21.15.6:0/2150171153 shutdown_connections 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 --2- 172.21.15.6:0/2150171153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80941055e0 0x7f809410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 --2- 172.21.15.6:0/2150171153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100540 0x7f8094100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 --2- 172.21.15.6:0/2150171153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8094100ee0 0x7f8094104eb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 -- 172.21.15.6:0/2150171153 >> 172.21.15.6:0/2150171153 conn(0x7f80940fbdc0 msgr2=0x7f80940fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.161+0000 7f809afd2700 1 -- 172.21.15.6:0/2150171153 shutdown_connections 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.162+0000 7f809afd2700 1 -- 172.21.15.6:0/2150171153 wait complete. 2023-11-17T00:07:07.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 Processor -- start 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 -- start start 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8094100540 0x7f80941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 0x7f8094198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80941055e0 0x7f809419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809419d5e0 con 0x7f80941055e0 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809419d750 con 0x7f8094100ee0 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.163+0000 7f809afd2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809419d910 con 0x7f8094100540 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f809956f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80941055e0 0x7f809419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 0x7f8094198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 0x7f8094198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59642/0 (socket says 172.21.15.6:59642) 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 -- 172.21.15.6:0/2702729666 learned_addr learned my addr 172.21.15.6:0/2702729666 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:07.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8098d6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8094100540 0x7f80941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 -- 172.21.15.6:0/2702729666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8094100540 msgr2=0x7f80941986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8094100540 0x7f80941986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 -- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80941055e0 msgr2=0x7f809419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80941055e0 0x7f809419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800095f0 con 0x7f8094100ee0 2023-11-17T00:07:07.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f809956f700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80941055e0 0x7f809419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:07.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.164+0000 7f8093fff700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 0x7f8094198c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8080009fd0 tx=0x7f808000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.165+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808001d070 con 0x7f8094100ee0 2023-11-17T00:07:07.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.165+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8094110360 con 0x7f8094100ee0 2023-11-17T00:07:07.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.165+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8080005050 con 0x7f8094100ee0 2023-11-17T00:07:07.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.165+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080004a90 con 0x7f8094100ee0 2023-11-17T00:07:07.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.165+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941101d0 con 0x7f8094100ee0 2023-11-17T00:07:07.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.166+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80940fd2a0 con 0x7f8094100ee0 2023-11-17T00:07:07.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.170+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8080005470 con 0x7f8094100ee0 2023-11-17T00:07:07.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8091ffb700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80840a11e0 0x7f80840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80800ce2a0 con 0x7f8094100ee0 2023-11-17T00:07:07.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8098d6e700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80840a11e0 0x7f80840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8091ffb700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] conn(0x7f80840ac1f0 0x7f80840ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:07.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80840aed50 con 0x7f80840ac1f0 2023-11-17T00:07:07.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.171+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80800ce610 con 0x7f8094100ee0 2023-11-17T00:07:07.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.172+0000 7f8098d6e700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80840a11e0 0x7f80840a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8088009c60 tx=0x7f8088009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.172+0000 7f809956f700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] conn(0x7f80840ac1f0 0x7f80840ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:07.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.173+0000 7f809956f700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] conn(0x7f80840ac1f0 0x7f80840ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:07.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.173+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== osd.1 v2:172.21.15.87:6800/4270277324 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f80840aed50 con 0x7f80840ac1f0 2023-11-17T00:07:07.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.190+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 --> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80941104f0 con 0x7f80840ac1f0 2023-11-17T00:07:07.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f8091ffb700 1 -- 172.21.15.6:0/2702729666 <== osd.1 v2:172.21.15.87:6800/4270277324 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80941104f0 con 0x7f80840ac1f0 2023-11-17T00:07:07.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] conn(0x7f80840ac1f0 msgr2=0x7f80840ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] conn(0x7f80840ac1f0 0x7f80840ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80840a11e0 msgr2=0x7f80840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80840a11e0 0x7f80840a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8088009c60 tx=0x7f8088009260 comp rx=0 tx=0).stop 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 msgr2=0x7f8094198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809afd2700 1 --2- 172.21.15.6:0/2702729666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8094100ee0 0x7f8094198c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8080009fd0 tx=0x7f808000f690 comp rx=0 tx=0).stop 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.191+0000 7f809956f700 1 -- 172.21.15.6:0/2702729666 reap_dead start 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.192+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 shutdown_connections 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.192+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 >> 172.21.15.6:0/2702729666 conn(0x7f80940fbdc0 msgr2=0x7f80940fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.192+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 shutdown_connections 2023-11-17T00:07:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:07.192+0000 7f809afd2700 1 -- 172.21.15.6:0/2702729666 wait complete. 2023-11-17T00:07:07.657 INFO:teuthology.orchestra.run.smithi006.stdout:219043332141 2023-11-17T00:07:07.658 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-11-17T00:07:07.886 INFO:teuthology.orchestra.run.smithi006.stdout:249108103207 2023-11-17T00:07:07.886 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-11-17T00:07:08.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 -- 172.21.15.6:0/1716293048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e0100ee0 msgr2=0x7f14e0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 --2- 172.21.15.6:0/1716293048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e0100ee0 0x7f14e0104eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f14d40099c0 tx=0x7f14d4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 -- 172.21.15.6:0/1716293048 shutdown_connections 2023-11-17T00:07:08.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 --2- 172.21.15.6:0/1716293048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e01055e0 0x7f14e010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 --2- 172.21.15.6:0/1716293048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14e0100540 0x7f14e0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 --2- 172.21.15.6:0/1716293048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e0100ee0 0x7f14e0104eb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 -- 172.21.15.6:0/1716293048 >> 172.21.15.6:0/1716293048 conn(0x7f14e00fbdc0 msgr2=0x7f14e00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 -- 172.21.15.6:0/1716293048 shutdown_connections 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.274+0000 7f14e6451700 1 -- 172.21.15.6:0/1716293048 wait complete. 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.275+0000 7f14e6451700 1 Processor -- start 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 -- start start 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 0x7f14e0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:08.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14e0100ee0 0x7f14e0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:08.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 0x7f14e0073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:08.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a8d80 con 0x7f14e01055e0 2023-11-17T00:07:08.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a8ec0 con 0x7f14e0100540 2023-11-17T00:07:08.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e6451700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a9000 con 0x7f14e0100ee0 2023-11-17T00:07:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e49ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 0x7f14e0073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 0x7f14e0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14dffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 0x7f14e0072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59670/0 (socket says 172.21.15.6:59670) 2023-11-17T00:07:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14dffff700 1 -- 172.21.15.6:0/8428852 learned_addr learned my addr 172.21.15.6:0/8428852 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:08.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.276+0000 7f14e49ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 0x7f14e0073870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56514/0 (socket says 172.21.15.6:56514) 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14df7fe700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14e0100ee0 0x7f14e0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 -- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14e0100ee0 msgr2=0x7f14e0073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14e0100ee0 0x7f14e0073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 -- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 msgr2=0x7f14e0073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 0x7f14e0073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40095f0 con 0x7f14e0100540 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14e49ee700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14e01055e0 0x7f14e0073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dffff700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 0x7f14e0072df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d980 tx=0x7f14d000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:08.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0004220 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.277+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e0110360 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.278+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d0007640 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.278+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d00056c0 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.278+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01101d0 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.278+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14e00fd1a0 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.281+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f14d0004380 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14dd7fa700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f14c80a11e0 0x7f14c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f14d00cd1d0 con 0x7f14e0100540 2023-11-17T00:07:08.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14df7fe700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f14c80a11e0 0x7f14c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14dd7fa700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] conn(0x7f14c80ac1f0 0x7f14c80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14c80aed50 con 0x7f14c80ac1f0 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.282+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d00cd540 con 0x7f14e0100540 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.283+0000 7f14df7fe700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f14c80a11e0 0x7f14c80a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006010 tx=0x7f14d4003680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.283+0000 7f14e49ee700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] conn(0x7f14c80ac1f0 0x7f14c80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:08.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.284+0000 7f14e49ee700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] conn(0x7f14c80ac1f0 0x7f14c80ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.284+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== osd.11 v2:172.21.15.186:6824/3407026982 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f14c80aed50 con 0x7f14c80ac1f0 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.302+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 --> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f14e01104f0 con 0x7f14c80ac1f0 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.303+0000 7f14dd7fa700 1 -- 172.21.15.6:0/8428852 <== osd.11 v2:172.21.15.186:6824/3407026982 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f14e01104f0 con 0x7f14c80ac1f0 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.303+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] conn(0x7f14c80ac1f0 msgr2=0x7f14c80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.303+0000 7f14e6451700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] conn(0x7f14c80ac1f0 0x7f14c80ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f14c80a11e0 msgr2=0x7f14c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f14c80a11e0 0x7f14c80a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006010 tx=0x7f14d4003680 comp rx=0 tx=0).stop 2023-11-17T00:07:08.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 msgr2=0x7f14e0072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:08.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 --2- 172.21.15.6:0/8428852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e0100540 0x7f14e0072df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d980 tx=0x7f14d000dc90 comp rx=0 tx=0).stop 2023-11-17T00:07:08.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e49ee700 1 -- 172.21.15.6:0/8428852 reap_dead start 2023-11-17T00:07:08.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 shutdown_connections 2023-11-17T00:07:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 >> 172.21.15.6:0/8428852 conn(0x7f14e00fbdc0 msgr2=0x7f14e00fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 shutdown_connections 2023-11-17T00:07:08.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:08.304+0000 7f14e6451700 1 -- 172.21.15.6:0/8428852 wait complete. 2023-11-17T00:07:09.094 INFO:teuthology.orchestra.run.smithi006.stdout:163208757303 2023-11-17T00:07:09.095 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-11-17T00:07:09.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:08 smithi087 ceph-mon[131182]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:08 smithi186 ceph-mon[131487]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:09.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:08 smithi006 ceph-mon[97840]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:09.610 INFO:teuthology.orchestra.run.smithi006.stdout:55834574924 2023-11-17T00:07:09.611 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-11-17T00:07:10.134 INFO:teuthology.orchestra.run.smithi006.stdout:304942678044 2023-11-17T00:07:10.134 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-11-17T00:07:10.259 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 -- 172.21.15.6:0/2403347874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 msgr2=0x7f6424111130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2403347874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f6424111130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6414009990 tx=0x7f6414009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 -- 172.21.15.6:0/2403347874 shutdown_connections 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2403347874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f642410b6f0 0x7f6424117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2403347874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f6424111130 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2403347874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410ad40 0x7f642410b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 -- 172.21.15.6:0/2403347874 >> 172.21.15.6:0/2403347874 conn(0x7f642406b3d0 msgr2=0x7f642406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 -- 172.21.15.6:0/2403347874 shutdown_connections 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.679+0000 7f642a3d1700 1 -- 172.21.15.6:0/2403347874 wait complete. 2023-11-17T00:07:10.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.680+0000 7f642a3d1700 1 Processor -- start 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.680+0000 7f642a3d1700 1 -- start start 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f642410ad40 0x7f64241a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410b6f0 0x7f64241a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f64241a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a5d30 con 0x7f642410ad40 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a5ea0 con 0x7f6424110d60 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642a3d1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a6060 con 0x7f642410b6f0 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f6423fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f642410ad40 0x7f64241a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f64237fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410b6f0 0x7f64241a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f64241a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f64237fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410b6f0 0x7f64241a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44982/0 (socket says 172.21.15.6:44982) 2023-11-17T00:07:10.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f64241a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55356/0 (socket says 172.21.15.6:55356) 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f64237fe700 1 -- 172.21.15.6:0/2366620632 learned_addr learned my addr 172.21.15.6:0/2366620632 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 -- 172.21.15.6:0/2366620632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410b6f0 msgr2=0x7f64241a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f642410b6f0 0x7f64241a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 -- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f642410ad40 msgr2=0x7f64241a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f642410ad40 0x7f64241a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.681+0000 7f642896e700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009520 con 0x7f6424110d60 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f642896e700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f64241a5720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f641c00efc0 tx=0x7f641c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c018070 con 0x7f6424110d60 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641c00f040 con 0x7f6424110d60 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64140095f0 con 0x7f6424110d60 2023-11-17T00:07:10.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00cd40 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.682+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642411ad80 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.683+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f642410a1a0 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.685+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f641c007500 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.686+0000 7f64217fa700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f640c0a1260 0x7f640c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f641c0cd2a0 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f6423fff700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f640c0a1260 0x7f640c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f64217fa700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] conn(0x7f640c0ac270 0x7f640c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f640c0aedd0 con 0x7f640c0ac270 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f641c0cd610 con 0x7f6424110d60 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.687+0000 7f64237fe700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] conn(0x7f640c0ac270 0x7f640c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.688+0000 7f6423fff700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f640c0a1260 0x7f640c0a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f641400b2e0 tx=0x7f6414005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:10.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.688+0000 7f64237fe700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] conn(0x7f640c0ac270 0x7f640c0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:10.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.689+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== osd.4 v2:172.21.15.87:6808/31913471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f640c0aedd0 con 0x7f640c0ac270 2023-11-17T00:07:10.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.706+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 --> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f642411abf0 con 0x7f640c0ac270 2023-11-17T00:07:10.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.706+0000 7f64217fa700 1 -- 172.21.15.6:0/2366620632 <== osd.4 v2:172.21.15.87:6808/31913471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f642411abf0 con 0x7f640c0ac270 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] conn(0x7f640c0ac270 msgr2=0x7f640c0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] conn(0x7f640c0ac270 0x7f640c0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f640c0a1260 msgr2=0x7f640c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f640c0a1260 0x7f640c0a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f641400b2e0 tx=0x7f6414005fd0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 msgr2=0x7f64241a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 --2- 172.21.15.6:0/2366620632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424110d60 0x7f64241a5720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f641c00efc0 tx=0x7f641c00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642896e700 1 -- 172.21.15.6:0/2366620632 reap_dead start 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 shutdown_connections 2023-11-17T00:07:10.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 >> 172.21.15.6:0/2366620632 conn(0x7f642406b3d0 msgr2=0x7f6424109790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:10.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 shutdown_connections 2023-11-17T00:07:10.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.707+0000 7f642a3d1700 1 -- 172.21.15.6:0/2366620632 wait complete. 2023-11-17T00:07:10.993 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:10 smithi006 ceph-mon[97840]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:10.994 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:10 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:07:11.033 INFO:teuthology.orchestra.run.smithi006.stdout:339302416408 2023-11-17T00:07:11.034 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-11-17T00:07:11.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 -- 172.21.15.6:0/2083128312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 msgr2=0x7fde08106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 --2- 172.21.15.6:0/2083128312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 0x7fde08106750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009990 tx=0x7fddf8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 -- 172.21.15.6:0/2083128312 shutdown_connections 2023-11-17T00:07:11.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 --2- 172.21.15.6:0/2083128312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde0810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 --2- 172.21.15.6:0/2083128312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 0x7fde08106750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 --2- 172.21.15.6:0/2083128312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 0x7fde08100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 -- 172.21.15.6:0/2083128312 >> 172.21.15.6:0/2083128312 conn(0x7fde08077530 msgr2=0x7fde08077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:11.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 -- 172.21.15.6:0/2083128312 shutdown_connections 2023-11-17T00:07:11.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.941+0000 7fde0e951700 1 -- 172.21.15.6:0/2083128312 wait complete. 2023-11-17T00:07:11.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.942+0000 7fde0e951700 1 Processor -- start 2023-11-17T00:07:11.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.942+0000 7fde0e951700 1 -- start start 2023-11-17T00:07:11.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 0x7fde08196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde081968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 0x7fde0819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0819b330 con 0x7fde08101400 2023-11-17T00:07:11.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0819b4a0 con 0x7fde08106380 2023-11-17T00:07:11.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0e951700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0819b660 con 0x7fde08100a50 2023-11-17T00:07:11.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde081968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde07fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 0x7fde08196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde0ceee700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 0x7fde0819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde081968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45590/0 (socket says 172.21.15.6:45590) 2023-11-17T00:07:11.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde07fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 0x7fde08196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44986/0 (socket says 172.21.15.6:44986) 2023-11-17T00:07:11.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.943+0000 7fde077fe700 1 -- 172.21.15.6:0/1683372049 learned_addr learned my addr 172.21.15.6:0/1683372049 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:11.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 -- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 msgr2=0x7fde08196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde08100a50 0x7fde08196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 -- 172.21.15.6:0/1683372049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 msgr2=0x7fde0819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde08106380 0x7fde0819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf80095f0 con 0x7fde08101400 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde077fe700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde081968b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d980 tx=0x7fddfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc004220 con 0x7fde08101400 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddfc007640 con 0x7fde08101400 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc005690 con 0x7fde08101400 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde08110350 con 0x7fde08101400 2023-11-17T00:07:11.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.944+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde081101c0 con 0x7fde08101400 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.945+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde08100150 con 0x7fde08101400 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.947+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fddfc004750 con 0x7fde08101400 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde057fa700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fddf00a1260 0x7fddf00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fddfc0cd200 con 0x7fde08101400 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde07fff700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fddf00a1260 0x7fddf00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde057fa700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] conn(0x7fddf00ac1f0 0x7fddf00ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddf00aed50 con 0x7fddf00ac1f0 2023-11-17T00:07:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddfc0cd570 con 0x7fde08101400 2023-11-17T00:07:11.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde07fff700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fddf00a1260 0x7fddf00a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b2e0 tx=0x7fddf8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.949+0000 7fde0ceee700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] conn(0x7fddf00ac1f0 0x7fddf00ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.950+0000 7fde0ceee700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] conn(0x7fddf00ac1f0 0x7fddf00ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.951+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== osd.2 v2:172.21.15.186:6800/1950482303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fddf00aed50 con 0x7fddf00ac1f0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.967+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 --> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde081104e0 con 0x7fddf00ac1f0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde057fa700 1 -- 172.21.15.6:0/1683372049 <== osd.2 v2:172.21.15.186:6800/1950482303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fde081104e0 con 0x7fddf00ac1f0 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] conn(0x7fddf00ac1f0 msgr2=0x7fddf00ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] conn(0x7fddf00ac1f0 0x7fddf00ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fddf00a1260 msgr2=0x7fddf00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fddf00a1260 0x7fddf00a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b2e0 tx=0x7fddf8005fd0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 msgr2=0x7fde081968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 --2- 172.21.15.6:0/1683372049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fde08101400 0x7fde081968b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d980 tx=0x7fddfc00dc90 comp rx=0 tx=0).stop 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0ceee700 1 -- 172.21.15.6:0/1683372049 reap_dead start 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 shutdown_connections 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 >> 172.21.15.6:0/1683372049 conn(0x7fde08077530 msgr2=0x7fde080ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.968+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 shutdown_connections 2023-11-17T00:07:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:10.969+0000 7fde0e951700 1 -- 172.21.15.6:0/1683372049 wait complete. 2023-11-17T00:07:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:10 smithi087 ceph-mon[131182]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:10 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:07:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:10 smithi186 ceph-mon[131487]: pgmap v324: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:10 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:07:11.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 -- 172.21.15.6:0/2476347855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 msgr2=0x7faf68104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 --2- 172.21.15.6:0/2476347855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf68104ec0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0099c0 tx=0x7faf5c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 -- 172.21.15.6:0/2476347855 shutdown_connections 2023-11-17T00:07:11.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 --2- 172.21.15.6:0/2476347855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf681055f0 0x7faf6810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 --2- 172.21.15.6:0/2476347855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf68100550 0x7faf68100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 --2- 172.21.15.6:0/2476347855 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf68104ec0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 -- 172.21.15.6:0/2476347855 >> 172.21.15.6:0/2476347855 conn(0x7faf680fbdf0 msgr2=0x7faf680fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.752+0000 7faf6e91f700 1 -- 172.21.15.6:0/2476347855 shutdown_connections 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.753+0000 7faf6e91f700 1 -- 172.21.15.6:0/2476347855 wait complete. 2023-11-17T00:07:11.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 Processor -- start 2023-11-17T00:07:11.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 -- start start 2023-11-17T00:07:11.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 0x7faf6810bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf6810c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf681055f0 0x7faf681a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a19c0 con 0x7faf68100ef0 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a1b00 con 0x7faf681055f0 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.754+0000 7faf6e91f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a1cc0 con 0x7faf68100550 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 0x7faf6810bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 0x7faf6810bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45000/0 (socket says 172.21.15.6:45000) 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 -- 172.21.15.6:0/3387523294 learned_addr learned my addr 172.21.15.6:0/3387523294 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf677fe700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf6810c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 -- 172.21.15.6:0/3387523294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf681055f0 msgr2=0x7faf681a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf681055f0 0x7faf681a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 -- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 msgr2=0x7faf6810c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf6810c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf67fff700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58009520 con 0x7faf68100550 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.755+0000 7faf677fe700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf68100ef0 0x7faf6810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf67fff700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 0x7faf6810bf90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf5800eab0 tx=0x7faf5800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800cc90 con 0x7faf68100550 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf580044e0 con 0x7faf68100550 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c0095f0 con 0x7faf68100550 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58010430 con 0x7faf68100550 2023-11-17T00:07:11.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.756+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68110370 con 0x7faf68100550 2023-11-17T00:07:11.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.757+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf680fd260 con 0x7faf68100550 2023-11-17T00:07:11.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.759+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7faf58003f40 con 0x7faf68100550 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf657fa700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7faf500a1260 0x7faf500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf58014070 con 0x7faf68100550 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf677fe700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7faf500a1260 0x7faf500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf657fa700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] conn(0x7faf500ac270 0x7faf500ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf500aedd0 con 0x7faf500ac270 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.761+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf580cd640 con 0x7faf68100550 2023-11-17T00:07:11.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.762+0000 7faf6cebc700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] conn(0x7faf500ac270 0x7faf500ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:11.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.762+0000 7faf677fe700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7faf500a1260 0x7faf500a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0098e0 tx=0x7faf5c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.762+0000 7faf6cebc700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] conn(0x7faf500ac270 0x7faf500ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:11.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.764+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== osd.9 v2:172.21.15.6:6826/1254890156 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faf500aedd0 con 0x7faf500ac270 2023-11-17T00:07:11.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.782+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 --> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf681101e0 con 0x7faf500ac270 2023-11-17T00:07:11.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf657fa700 1 -- 172.21.15.6:0/3387523294 <== osd.9 v2:172.21.15.6:6826/1254890156 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faf681101e0 con 0x7faf500ac270 2023-11-17T00:07:11.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] conn(0x7faf500ac270 msgr2=0x7faf500ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] conn(0x7faf500ac270 0x7faf500ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7faf500a1260 msgr2=0x7faf500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7faf500a1260 0x7faf500a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0098e0 tx=0x7faf5c0058e0 comp rx=0 tx=0).stop 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 msgr2=0x7faf6810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.783+0000 7faf6e91f700 1 --2- 172.21.15.6:0/3387523294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf68100550 0x7faf6810bf90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf5800eab0 tx=0x7faf5800ee70 comp rx=0 tx=0).stop 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.784+0000 7faf6cebc700 1 -- 172.21.15.6:0/3387523294 reap_dead start 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.784+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 shutdown_connections 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.784+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 >> 172.21.15.6:0/3387523294 conn(0x7faf680fbdf0 msgr2=0x7faf680fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.784+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 shutdown_connections 2023-11-17T00:07:11.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:11.784+0000 7faf6e91f700 1 -- 172.21.15.6:0/3387523294 wait complete. 2023-11-17T00:07:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:11.979 INFO:teuthology.orchestra.run.smithi006.stdout:141733920828 2023-11-17T00:07:11.980 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-11-17T00:07:12.341 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:12.381 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:12.572 INFO:teuthology.orchestra.run.smithi006.stdout:283467841570 2023-11-17T00:07:12.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-11-17T00:07:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:12 smithi006 ceph-mon[97840]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:12.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:12 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:12 smithi186 ceph-mon[131487]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:12 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.305 INFO:teuthology.orchestra.run.smithi006.stdout:77309411399 2023-11-17T00:07:13.305 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-11-17T00:07:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:12 smithi087 ceph-mon[131182]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-11-17T00:07:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:12 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 -- 172.21.15.6:0/4254019635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd40072e90 msgr2=0x7efd40073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 --2- 172.21.15.6:0/4254019635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd40072e90 0x7efd40073260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd300098f0 tx=0x7efd30009c00 comp rx=0 tx=0).stop 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 -- 172.21.15.6:0/4254019635 shutdown_connections 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 --2- 172.21.15.6:0/4254019635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd4010cb20 0x7efd4010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 --2- 172.21.15.6:0/4254019635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd40072e90 0x7efd40073260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 --2- 172.21.15.6:0/4254019635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd400737a0 0x7efd4010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 -- 172.21.15.6:0/4254019635 >> 172.21.15.6:0/4254019635 conn(0x7efd400fbdc0 msgr2=0x7efd400fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:13.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.882+0000 7efd4659e700 1 -- 172.21.15.6:0/4254019635 shutdown_connections 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.883+0000 7efd4659e700 1 -- 172.21.15.6:0/4254019635 wait complete. 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 Processor -- start 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 -- start start 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 0x7efd40198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4010cb20 0x7efd4019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019d5d0 con 0x7efd40072e90 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019d740 con 0x7efd400737a0 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd4659e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019d900 con 0x7efd4010cb20 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.884+0000 7efd3ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 0x7efd40198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45632/0 (socket says 172.21.15.6:45632) 2023-11-17T00:07:13.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 -- 172.21.15.6:0/3423288268 learned_addr learned my addr 172.21.15.6:0/3423288268 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 0x7efd40198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55412/0 (socket says 172.21.15.6:55412) 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd44b3b700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4010cb20 0x7efd4019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 -- 172.21.15.6:0/3423288268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4010cb20 msgr2=0x7efd4019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4010cb20 0x7efd4019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 -- 172.21.15.6:0/3423288268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 msgr2=0x7efd40198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 0x7efd40198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 -- 172.21.15.6:0/3423288268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd30009520 con 0x7efd40072e90 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.885+0000 7efd3ffff700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd30009810 tx=0x7efd3000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.886+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3001c070 con 0x7efd40072e90 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.886+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3000bb20 con 0x7efd40072e90 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.886+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40112550 con 0x7efd40072e90 2023-11-17T00:07:13.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.886+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30004930 con 0x7efd40072e90 2023-11-17T00:07:13.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.886+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd401123c0 con 0x7efd40072e90 2023-11-17T00:07:13.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.887+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd401126e0 con 0x7efd40072e90 2023-11-17T00:07:13.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.889+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efd30020070 con 0x7efd40072e90 2023-11-17T00:07:13.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.890+0000 7efd3d7fa700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 0x7efd280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:13.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.891+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efd300cda80 con 0x7efd40072e90 2023-11-17T00:07:13.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.891+0000 7efd3f7fe700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 0x7efd280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:13.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.891+0000 7efd3f7fe700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 0x7efd280a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd40199ba0 tx=0x7efd34009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:13.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:13.895+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efd30096d90 con 0x7efd40072e90 2023-11-17T00:07:14.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.137+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efd40066ea0 con 0x7efd40072e90 2023-11-17T00:07:14.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.138+0000 7efd3d7fa700 1 -- 172.21.15.6:0/3423288268 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd3009a3b0 con 0x7efd40072e90 2023-11-17T00:07:14.139 INFO:teuthology.orchestra.run.smithi006.stdout:197568495666 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 msgr2=0x7efd280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 0x7efd280a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd40199ba0 tx=0x7efd34009260 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 msgr2=0x7efd40198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd30009810 tx=0x7efd3000fa00 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 shutdown_connections 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efd280a11e0 0x7efd280a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efd40072e90 0x7efd40198580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4010cb20 0x7efd4019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 --2- 172.21.15.6:0/3423288268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd400737a0 0x7efd40198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 >> 172.21.15.6:0/3423288268 conn(0x7efd400fbdc0 msgr2=0x7efd4010ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 shutdown_connections 2023-11-17T00:07:14.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:14.140+0000 7efd4659e700 1 -- 172.21.15.6:0/3423288268 wait complete. 2023-11-17T00:07:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:14 smithi006 ceph-mon[97840]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:14.472 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:14 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3423288268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-17T00:07:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:14 smithi087 ceph-mon[131182]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:14 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3423288268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-17T00:07:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:14 smithi186 ceph-mon[131487]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:14 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3423288268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-17T00:07:14.886 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:14.887 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:15.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495666 for osd.6 2023-11-17T00:07:15.333 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.596+0000 7f862b4ac700 1 -- 172.21.15.6:0/2878858410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86241112f0 msgr2=0x7f86241136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.596+0000 7f862b4ac700 1 --2- 172.21.15.6:0/2878858410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86241112f0 0x7f86241136d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b0d0 tx=0x7f861c00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 -- 172.21.15.6:0/2878858410 shutdown_connections 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 --2- 172.21.15.6:0/2878858410 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8624072e90 0x7f8624073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 --2- 172.21.15.6:0/2878858410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86241112f0 0x7f86241136d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 --2- 172.21.15.6:0/2878858410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86240737a0 0x7f8624110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 -- 172.21.15.6:0/2878858410 >> 172.21.15.6:0/2878858410 conn(0x7f86240fbdc0 msgr2=0x7f86240fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 -- 172.21.15.6:0/2878858410 shutdown_connections 2023-11-17T00:07:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.597+0000 7f862b4ac700 1 -- 172.21.15.6:0/2878858410 wait complete. 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.598+0000 7f862b4ac700 1 Processor -- start 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.598+0000 7f862b4ac700 1 -- start start 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 0x7f86241a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 0x7f86241a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 0x7f862419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f8629248700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 0x7f86241a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862419ce00 con 0x7f86241112f0 2023-11-17T00:07:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862419cf70 con 0x7f86240737a0 2023-11-17T00:07:15.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f862b4ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862419d130 con 0x7f8624072e90 2023-11-17T00:07:15.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f8629248700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 0x7f86241a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45034/0 (socket says 172.21.15.6:45034) 2023-11-17T00:07:15.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.599+0000 7f8629248700 1 -- 172.21.15.6:0/3865226988 learned_addr learned my addr 172.21.15.6:0/3865226988 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:15.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8629a49700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 0x7f862419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 0x7f86241a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 -- 172.21.15.6:0/3865226988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 msgr2=0x7f86241a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 0x7f86241a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 -- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 msgr2=0x7f862419c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 0x7f862419c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8628a47700 1 -- 172.21.15.6:0/3865226988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c009d00 con 0x7f86240737a0 2023-11-17T00:07:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.600+0000 7f8629a49700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 0x7f862419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.601+0000 7f8628a47700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 0x7f86241a2c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f861400e9f0 tx=0x7f861400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.601+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861400eee0 con 0x7f86240737a0 2023-11-17T00:07:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.602+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8624116900 con 0x7f86240737a0 2023-11-17T00:07:15.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.602+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8624116770 con 0x7f86240737a0 2023-11-17T00:07:15.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.602+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861400cbd0 con 0x7f86240737a0 2023-11-17T00:07:15.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.602+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614018680 con 0x7f86240737a0 2023-11-17T00:07:15.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.604+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8608005320 con 0x7f86240737a0 2023-11-17T00:07:15.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.608+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f861400cd40 con 0x7f86240737a0 2023-11-17T00:07:15.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.611+0000 7f861a7fc700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 0x7f86100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.611+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8614014070 con 0x7f86240737a0 2023-11-17T00:07:15.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.611+0000 7f8629248700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 0x7f86100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.612+0000 7f8629248700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 0x7f86100a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8620009cb0 tx=0x7f8620009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:15.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.614+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8614051730 con 0x7f86240737a0 2023-11-17T00:07:15.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.867+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8608005cc0 con 0x7f86240737a0 2023-11-17T00:07:15.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.868+0000 7f861a7fc700 1 -- 172.21.15.6:0/3865226988 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86140d3080 con 0x7f86240737a0 2023-11-17T00:07:15.869 INFO:teuthology.orchestra.run.smithi006.stdout:219043332143 2023-11-17T00:07:15.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.871+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 msgr2=0x7f86100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.871+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 0x7f86100a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8620009cb0 tx=0x7f8620009260 comp rx=0 tx=0).stop 2023-11-17T00:07:15.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.871+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 msgr2=0x7f86241a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.871+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 0x7f86241a2c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f861400e9f0 tx=0x7f861400ed00 comp rx=0 tx=0).stop 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 shutdown_connections 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86100a1260 0x7f86100a3710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86241112f0 0x7f862419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8624072e90 0x7f86241a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 --2- 172.21.15.6:0/3865226988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86240737a0 0x7f86241a2c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 >> 172.21.15.6:0/3865226988 conn(0x7f86240fbdc0 msgr2=0x7f86240fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 shutdown_connections 2023-11-17T00:07:15.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.872+0000 7f862b4ac700 1 -- 172.21.15.6:0/3865226988 wait complete. 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/2356586556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 msgr2=0x7fdbf4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/2356586556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf4104eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009990 tx=0x7fdbf0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/2356586556 shutdown_connections 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/2356586556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf41055e0 0x7fdbf410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/2356586556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4100540 0x7fdbf4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/2356586556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf4104eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/2356586556 >> 172.21.15.6:0/2356586556 conn(0x7fdbf40fbdc0 msgr2=0x7fdbf40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:15.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/2356586556 shutdown_connections 2023-11-17T00:07:15.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.918+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/2356586556 wait complete. 2023-11-17T00:07:15.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 Processor -- start 2023-11-17T00:07:15.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 -- start start 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 0x7fdbf41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 0x7fdbf4109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf410a380 con 0x7fdbf4100540 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf410a4f0 con 0x7fdbf4100ee0 2023-11-17T00:07:15.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.920+0000 7fdbfbe2f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf410a6b0 con 0x7fdbf41055e0 2023-11-17T00:07:15.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbf93ca700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbf93ca700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55440/0 (socket says 172.21.15.6:55440) 2023-11-17T00:07:15.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbf93ca700 1 -- 172.21.15.6:0/748160122 learned_addr learned my addr 172.21.15.6:0/748160122 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:15.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbfa3cc700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 0x7fdbf4109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbf9bcb700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 0x7fdbf41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.921+0000 7fdbf93ca700 1 -- 172.21.15.6:0/748160122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 msgr2=0x7fdbf4109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf93ca700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 0x7fdbf4109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf93ca700 1 -- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 msgr2=0x7fdbf41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf93ca700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 0x7fdbf41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf93ca700 1 -- 172.21.15.6:0/748160122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00095f0 con 0x7fdbf4100ee0 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbfa3cc700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 0x7fdbf4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:15.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf9bcb700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 0x7fdbf41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbf93ca700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.922+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf001d070 con 0x7fdbf4100ee0 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.923+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf4110360 con 0x7fdbf4100ee0 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.923+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf0005580 con 0x7fdbf4100ee0 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.923+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf41101d0 con 0x7fdbf4100ee0 2023-11-17T00:07:15.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.924+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf000fd90 con 0x7fdbf4100ee0 2023-11-17T00:07:15.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.924+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf41104f0 con 0x7fdbf4100ee0 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.927+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdbf0005050 con 0x7fdbf4100ee0 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.929+0000 7fdbeaffd700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 0x7fdbe00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.930+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdbf00cdee0 con 0x7fdbf4100ee0 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.930+0000 7fdbf9bcb700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 0x7fdbe00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.931+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbf0097250 con 0x7fdbf4100ee0 2023-11-17T00:07:15.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:15.931+0000 7fdbf9bcb700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 0x7fdbe00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400afd0 tx=0x7fdbe400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:16.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.176+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdbf410b180 con 0x7fdbf4100ee0 2023-11-17T00:07:16.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.177+0000 7fdbeaffd700 1 -- 172.21.15.6:0/748160122 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdbf009a870 con 0x7fdbf4100ee0 2023-11-17T00:07:16.181 INFO:teuthology.orchestra.run.smithi006.stdout:34359738449 2023-11-17T00:07:16.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 msgr2=0x7fdbe00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 0x7fdbe00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400afd0 tx=0x7fdbe400a380 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 msgr2=0x7fdbf41a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0003b90 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 shutdown_connections 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbe00a11e0 0x7fdbe00a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbf4100540 0x7fdbf41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf41055e0 0x7fdbf4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 --2- 172.21.15.6:0/748160122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbf4100ee0 0x7fdbf41a1c60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 >> 172.21.15.6:0/748160122 conn(0x7fdbf40fbdc0 msgr2=0x7fdbf410b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 shutdown_connections 2023-11-17T00:07:16.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:16.180+0000 7fdbfbe2f700 1 -- 172.21.15.6:0/748160122 wait complete. 2023-11-17T00:07:16.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332143 for osd.7 2023-11-17T00:07:16.626 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:16.953 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:16 smithi006 ceph-mon[97840]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:16.953 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3865226988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-17T00:07:16.953 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/748160122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-17T00:07:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:16 smithi087 ceph-mon[131182]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3865226988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-17T00:07:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/748160122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-17T00:07:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:16 smithi186 ceph-mon[131487]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3865226988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-17T00:07:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/748160122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-17T00:07:17.118 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:17.268 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 -- 172.21.15.6:0/1296943967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 msgr2=0x7fdd2810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/1296943967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd2810d160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b0d0 tx=0x7fdd2000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 -- 172.21.15.6:0/1296943967 shutdown_connections 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/1296943967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd2810d160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/1296943967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/1296943967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 0x7fdd28104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 -- 172.21.15.6:0/1296943967 >> 172.21.15.6:0/1296943967 conn(0x7fdd280fbdc0 msgr2=0x7fdd280fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 -- 172.21.15.6:0/1296943967 shutdown_connections 2023-11-17T00:07:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.401+0000 7fdd2ceea700 1 -- 172.21.15.6:0/1296943967 wait complete. 2023-11-17T00:07:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.402+0000 7fdd2ceea700 1 Processor -- start 2023-11-17T00:07:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.402+0000 7fdd2ceea700 1 -- start start 2023-11-17T00:07:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 0x7fdd28073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:17.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28073c20 con 0x7fdd281055e0 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd281a4b80 con 0x7fdd28100540 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2ceea700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd281a4cf0 con 0x7fdd28100ee0 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd2659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:17.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45688/0 (socket says 172.21.15.6:45688) 2023-11-17T00:07:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.403+0000 7fdd26d9d700 1 -- 172.21.15.6:0/2253348561 learned_addr learned my addr 172.21.15.6:0/2253348561 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:17.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd25d9b700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 0x7fdd28073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 -- 172.21.15.6:0/2253348561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 msgr2=0x7fdd28073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 0x7fdd28073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 -- 172.21.15.6:0/2253348561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 msgr2=0x7fdd28072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 -- 172.21.15.6:0/2253348561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009d00 con 0x7fdd281055e0 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd2659c700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:17.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd26d9d700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd20007870 tx=0x7fdd200078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000e040 con 0x7fdd281055e0 2023-11-17T00:07:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd20005070 con 0x7fdd281055e0 2023-11-17T00:07:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.404+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2001b6c0 con 0x7fdd281055e0 2023-11-17T00:07:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.405+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd28110320 con 0x7fdd281055e0 2023-11-17T00:07:17.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.405+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd28110190 con 0x7fdd281055e0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.406+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd281104b0 con 0x7fdd281055e0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.408+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdd200051e0 con 0x7fdd281055e0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.409+0000 7fdd177fe700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 0x7fdd100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.409+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdd200cdea0 con 0x7fdd281055e0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.410+0000 7fdd2659c700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 0x7fdd100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.410+0000 7fdd2659c700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 0x7fdd100a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd18011490 tx=0x7fdd18009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.413+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdd200971b0 con 0x7fdd281055e0 2023-11-17T00:07:17.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.631+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdd28066ea0 con 0x7fdd281055e0 2023-11-17T00:07:17.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.631+0000 7fdd177fe700 1 -- 172.21.15.6:0/2253348561 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd2009a7d0 con 0x7fdd281055e0 2023-11-17T00:07:17.633 INFO:teuthology.orchestra.run.smithi006.stdout:107374182465 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 msgr2=0x7fdd100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 0x7fdd100a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd18011490 tx=0x7fdd18009260 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 msgr2=0x7fdd281a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd20007870 tx=0x7fdd200078a0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 shutdown_connections 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdd100a1260 0x7fdd100a3710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdd281055e0 0x7fdd281a4640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd28100540 0x7fdd28072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 --2- 172.21.15.6:0/2253348561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd28100ee0 0x7fdd28073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 >> 172.21.15.6:0/2253348561 conn(0x7fdd280fbdc0 msgr2=0x7fdd280fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 shutdown_connections 2023-11-17T00:07:17.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:17.634+0000 7fdd2ceea700 1 -- 172.21.15.6:0/2253348561 wait complete. 2023-11-17T00:07:17.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2253348561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-17T00:07:18.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2253348561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-17T00:07:18.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2253348561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-17T00:07:18.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 -- 172.21.15.6:0/2623174579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100ee0 msgr2=0x7f3c2c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 --2- 172.21.15.6:0/2623174579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c104eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0099c0 tx=0x7f3c1c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 -- 172.21.15.6:0/2623174579 shutdown_connections 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 --2- 172.21.15.6:0/2623174579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 --2- 172.21.15.6:0/2623174579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100540 0x7f3c2c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 --2- 172.21.15.6:0/2623174579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c104eb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.368+0000 7f3c334b6700 1 -- 172.21.15.6:0/2623174579 >> 172.21.15.6:0/2623174579 conn(0x7f3c2c0fbdc0 msgr2=0x7f3c2c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.369+0000 7f3c334b6700 1 -- 172.21.15.6:0/2623174579 shutdown_connections 2023-11-17T00:07:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.369+0000 7f3c334b6700 1 -- 172.21.15.6:0/2623174579 wait complete. 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 Processor -- start 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 -- start start 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c19d5a0 con 0x7f3c2c100540 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c19d710 con 0x7f3c2c1055e0 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.370+0000 7f3c334b6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c19d8d0 con 0x7f3c2c100ee0 2023-11-17T00:07:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45704/0 (socket says 172.21.15.6:45704) 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 -- 172.21.15.6:0/1395603214 learned_addr learned my addr 172.21.15.6:0/1395603214 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31a53700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c30a51700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 -- 172.21.15.6:0/1395603214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100ee0 msgr2=0x7f3c2c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 -- 172.21.15.6:0/1395603214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 msgr2=0x7f3c2c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 -- 172.21.15.6:0/1395603214 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0095f0 con 0x7f3c2c100540 2023-11-17T00:07:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31a53700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.371+0000 7f3c31252700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e9a0 tx=0x7f3c2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.372+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28009790 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.372+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c28010430 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.372+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28018680 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.372+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c110360 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.372+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c1101d0 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.373+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c1104f0 con 0x7f3c2c100540 2023-11-17T00:07:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.375+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3c280105a0 con 0x7f3c2c100540 2023-11-17T00:07:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.376+0000 7f3c227fc700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 0x7f3c180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.377+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c28014070 con 0x7f3c2c100540 2023-11-17T00:07:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.377+0000 7f3c30a51700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 0x7f3c180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:18.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.377+0000 7f3c30a51700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 0x7f3c180a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0098e0 tx=0x7f3c1c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.380+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3c28051730 con 0x7f3c2c100540 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.599+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3c2c1a5660 con 0x7f3c2c100540 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.600+0000 7f3c227fc700 1 -- 172.21.15.6:0/1395603214 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c28096200 con 0x7f3c2c100540 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 msgr2=0x7f3c180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 0x7f3c180a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0098e0 tx=0x7f3c1c000bc0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 msgr2=0x7f3c2c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e9a0 tx=0x7f3c2800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 shutdown_connections 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3c180a1260 0x7f3c180a3710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3c2c100540 0x7f3c2c1985e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2c1055e0 0x7f3c2c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 --2- 172.21.15.6:0/1395603214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c2c100ee0 0x7f3c2c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 >> 172.21.15.6:0/1395603214 conn(0x7f3c2c0fbdc0 msgr2=0x7f3c2c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.602+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 shutdown_connections 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:18.603+0000 7f3c334b6700 1 -- 172.21.15.6:0/1395603214 wait complete. 2023-11-17T00:07:18.608 INFO:teuthology.orchestra.run.smithi006.stdout:163208757305 2023-11-17T00:07:18.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:18 smithi006 ceph-mon[97840]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:18.888 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1395603214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-17T00:07:19.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2023-11-17T00:07:19.033 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:18 smithi087 ceph-mon[131182]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:19.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1395603214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-17T00:07:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:18 smithi186 ceph-mon[131487]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-11-17T00:07:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1395603214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-17T00:07:19.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2023-11-17T00:07:19.850 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:19.877 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:20.079 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:20.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757303 got 163208757305 for osd.5 2023-11-17T00:07:20.337 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:20 smithi006 ceph-mon[97840]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-11-17T00:07:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:20 smithi087 ceph-mon[131182]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-11-17T00:07:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:20 smithi186 ceph-mon[131487]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-11-17T00:07:22.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 -- 172.21.15.6:0/1383163175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100ee0 msgr2=0x7fb4a4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:22.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/1383163175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100ee0 0x7fb4a4104eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4980099c0 tx=0x7fb498009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 -- 172.21.15.6:0/1383163175 shutdown_connections 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/1383163175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/1383163175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100540 0x7fb4a4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/1383163175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100ee0 0x7fb4a4104eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 -- 172.21.15.6:0/1383163175 >> 172.21.15.6:0/1383163175 conn(0x7fb4a40fbdc0 msgr2=0x7fb4a40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 -- 172.21.15.6:0/1383163175 shutdown_connections 2023-11-17T00:07:22.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.811+0000 7fb4a8cae700 1 -- 172.21.15.6:0/1383163175 wait complete. 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.812+0000 7fb4a8cae700 1 Processor -- start 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 -- start start 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 0x7fb4a4196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a419b2c0 con 0x7fb4a41055e0 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a419b430 con 0x7fb4a4100ee0 2023-11-17T00:07:22.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a8cae700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a419b5f0 con 0x7fb4a4100540 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.813+0000 7fb4a1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:53356/0 (socket says 172.21.15.6:53356) 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 -- 172.21.15.6:0/440111473 learned_addr learned my addr 172.21.15.6:0/440111473 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a259c700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 0x7fb4a4196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a2d9d700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 -- 172.21.15.6:0/440111473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 msgr2=0x7fb4a4196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 0x7fb4a4196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 -- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 msgr2=0x7fb4a419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 -- 172.21.15.6:0/440111473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4980095f0 con 0x7fb4a4100ee0 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a2d9d700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:22.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a259c700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 0x7fb4a4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.814+0000 7fb4a1d9b700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b580 tx=0x7fb49800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.815+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49801d070 con 0x7fb4a4100ee0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.815+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb498005050 con 0x7fb4a4100ee0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.815+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498004a90 con 0x7fb4a4100ee0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.815+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a4110310 con 0x7fb4a4100ee0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.815+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a4110180 con 0x7fb4a4100ee0 2023-11-17T00:07:22.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.816+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a419bb30 con 0x7fb4a4100ee0 2023-11-17T00:07:22.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.820+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb4980051c0 con 0x7fb4a4100ee0 2023-11-17T00:07:22.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.822+0000 7fb4937fe700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 0x7fb48c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.822+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb4980cdf00 con 0x7fb4a4100ee0 2023-11-17T00:07:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.822+0000 7fb4a259c700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 0x7fb48c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.822+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb498097210 con 0x7fb4a4100ee0 2023-11-17T00:07:22.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.823+0000 7fb4a259c700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 0x7fb48c0a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb494011490 tx=0x7fb494009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:22 smithi006 ceph-mon[97840]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1180678733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-17T00:07:23.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.040+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb4a41104a0 con 0x7fb4a4100ee0 2023-11-17T00:07:23.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.041+0000 7fb4937fe700 1 -- 172.21.15.6:0/440111473 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb49809a830 con 0x7fb4a4100ee0 2023-11-17T00:07:23.042 INFO:teuthology.orchestra.run.smithi006.stdout:249108103210 2023-11-17T00:07:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 msgr2=0x7fb48c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 0x7fb48c0a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb494011490 tx=0x7fb494009260 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 msgr2=0x7fb4a41968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b580 tx=0x7fb49800bfd0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 shutdown_connections 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb48c0a1260 0x7fb48c0a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4a41055e0 0x7fb4a419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a4100540 0x7fb4a4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 --2- 172.21.15.6:0/440111473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4100ee0 0x7fb4a41968b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 >> 172.21.15.6:0/440111473 conn(0x7fb4a40fbdc0 msgr2=0x7fb4a40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 shutdown_connections 2023-11-17T00:07:23.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.043+0000 7fb4a8cae700 1 -- 172.21.15.6:0/440111473 wait complete. 2023-11-17T00:07:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:22 smithi087 ceph-mon[131182]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:22 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1180678733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-17T00:07:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:22 smithi186 ceph-mon[131487]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:22 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1180678733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-17T00:07:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 -- 172.21.15.6:0/1493812726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 msgr2=0x7f64f8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1493812726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8104eb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f64e40099c0 tx=0x7f64e4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 -- 172.21.15.6:0/1493812726 shutdown_connections 2023-11-17T00:07:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1493812726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f81055e0 0x7f64f810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1493812726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f8100540 0x7f64f8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1493812726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8104eb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 -- 172.21.15.6:0/1493812726 >> 172.21.15.6:0/1493812726 conn(0x7f64f80fbdc0 msgr2=0x7f64f80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 -- 172.21.15.6:0/1493812726 shutdown_connections 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.355+0000 7f64fecfc700 1 -- 172.21.15.6:0/1493812726 wait complete. 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.356+0000 7f64fecfc700 1 Processor -- start 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.356+0000 7f64fecfc700 1 -- start start 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f8100540 0x7f64f81963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f81055e0 0x7f64f819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f819b390 con 0x7f64f8100ee0 2023-11-17T00:07:23.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f819b500 con 0x7f64f8100540 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fecfc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f819b6c0 con 0x7f64f81055e0 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38042/0 (socket says 172.21.15.6:38042) 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 -- 172.21.15.6:0/1180678733 learned_addr learned my addr 172.21.15.6:0/1180678733 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:55834574926 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fca98700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f8100540 0x7f64f81963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64fd299700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f81055e0 0x7f64f819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 -- 172.21.15.6:0/1180678733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f81055e0 msgr2=0x7f64f819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f81055e0 0x7f64f819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 -- 172.21.15.6:0/1180678733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f8100540 msgr2=0x7f64f81963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f8100540 0x7f64f81963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.357+0000 7f64f7fff700 1 -- 172.21.15.6:0/1180678733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40095f0 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64f7fff700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009f50 tx=0x7f64e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e401d070 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f8110360 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e4003f90 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400fcf0 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.358+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f81101d0 con 0x7f64f8100ee0 2023-11-17T00:07:23.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.359+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f81104f0 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.361+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f64e400f460 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.362+0000 7f64f5ffb700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 0x7f64e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.363+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64e40cdc90 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.363+0000 7f64fca98700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 0x7f64e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.363+0000 7f64fca98700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 0x7f64e80a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005950 tx=0x7f64ec00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.366+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f64e4096fa0 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.577+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f64f81a3450 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.577+0000 7f64f5ffb700 1 -- 172.21.15.6:0/1180678733 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64e409a5c0 con 0x7f64f8100ee0 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.579+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 msgr2=0x7f64e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.579+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 0x7f64e80a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005950 tx=0x7f64ec00b410 comp rx=0 tx=0).stop 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.579+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 msgr2=0x7f64f8196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.579+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009f50 tx=0x7f64e4005740 comp rx=0 tx=0).stop 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 shutdown_connections 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f64e80a11e0 0x7f64e80a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f81055e0 0x7f64f819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f8100540 0x7f64f81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 --2- 172.21.15.6:0/1180678733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f64f8100ee0 0x7f64f8196910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 >> 172.21.15.6:0/1180678733 conn(0x7f64f80fbdc0 msgr2=0x7f64f80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 shutdown_connections 2023-11-17T00:07:23.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:22.580+0000 7f64fecfc700 1 -- 172.21.15.6:0/1180678733 wait complete. 2023-11-17T00:07:23.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 -- 172.21.15.6:0/1179566146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5290100ef0 msgr2=0x7f5290104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 --2- 172.21.15.6:0/1179566146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5290100ef0 0x7f5290104ec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5280009990 tx=0x7f5280009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 -- 172.21.15.6:0/1179566146 shutdown_connections 2023-11-17T00:07:23.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 --2- 172.21.15.6:0/1179566146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52901055f0 0x7f529010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 --2- 172.21.15.6:0/1179566146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100550 0x7f5290100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 --2- 172.21.15.6:0/1179566146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5290100ef0 0x7f5290104ec0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 -- 172.21.15.6:0/1179566146 >> 172.21.15.6:0/1179566146 conn(0x7f52900fbdf0 msgr2=0x7f52900fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:23.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 -- 172.21.15.6:0/1179566146 shutdown_connections 2023-11-17T00:07:23.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.690+0000 7f5297a29700 1 -- 172.21.15.6:0/1179566146 wait complete. 2023-11-17T00:07:23.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 Processor -- start 2023-11-17T00:07:23.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 -- start start 2023-11-17T00:07:23.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100ef0 0x7f5290198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52901055f0 0x7f529019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529019d540 con 0x7f5290100550 2023-11-17T00:07:23.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529019d6b0 con 0x7f52901055f0 2023-11-17T00:07:23.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.691+0000 7f5297a29700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529019d870 con 0x7f5290100ef0 2023-11-17T00:07:23.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f5294fc4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100ef0 0x7f5290198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38064/0 (socket says 172.21.15.6:38064) 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 -- 172.21.15.6:0/2143694854 learned_addr learned my addr 172.21.15.6:0/2143694854 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f5295fc6700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52901055f0 0x7f529019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 -- 172.21.15.6:0/2143694854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100ef0 msgr2=0x7f5290198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100ef0 0x7f5290198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 -- 172.21.15.6:0/2143694854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52901055f0 msgr2=0x7f529019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52901055f0 0x7f529019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 -- 172.21.15.6:0/2143694854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009520 con 0x7f5290100550 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.692+0000 7f52957c5700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eb30 tx=0x7f528c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.693+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c009880 con 0x7f5290100550 2023-11-17T00:07:23.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.693+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52800095f0 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.693+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528c0044e0 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.693+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c003d80 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.693+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5290110320 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.694+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5290110190 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.696+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f528c003f20 con 0x7f5290100550 2023-11-17T00:07:23.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.697+0000 7f52867fc700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 0x7f527c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:23.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.697+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f528c014070 con 0x7f5290100550 2023-11-17T00:07:23.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.697+0000 7f5294fc4700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 0x7f527c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:23.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.698+0000 7f5294fc4700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 0x7f527c0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5290199ba0 tx=0x7f5280005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:23.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.701+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f528c0965c0 con 0x7f5290100550 2023-11-17T00:07:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.940+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5290066ea0 con 0x7f5290100550 2023-11-17T00:07:23.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.940+0000 7f52867fc700 1 -- 172.21.15.6:0/2143694854 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f528c099be0 con 0x7f5290100550 2023-11-17T00:07:23.994 INFO:teuthology.orchestra.run.smithi006.stdout:141733920830 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.942+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 msgr2=0x7f527c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.942+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 0x7f527c0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5290199ba0 tx=0x7f5280005fd0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.942+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 msgr2=0x7f5290198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.942+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eb30 tx=0x7f528c00ee40 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 shutdown_connections 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f527c0a11e0 0x7f527c0a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5290100550 0x7f5290198580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52901055f0 0x7f529019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 --2- 172.21.15.6:0/2143694854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5290100ef0 0x7f5290198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 >> 172.21.15.6:0/2143694854 conn(0x7f52900fbdf0 msgr2=0x7f52900fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 shutdown_connections 2023-11-17T00:07:23.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:23.943+0000 7f5297a29700 1 -- 172.21.15.6:0/2143694854 wait complete. 2023-11-17T00:07:23.996 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:23 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/440111473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-17T00:07:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:23 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/440111473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-17T00:07:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:23 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/440111473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-17T00:07:24.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103210 for osd.8 2023-11-17T00:07:24.905 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:24.953 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574926 for osd.1 2023-11-17T00:07:24.953 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:24.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:24 smithi006 ceph-mon[97840]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:24.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:24 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2143694854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-17T00:07:25.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 -- 172.21.15.6:0/1145957968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100ee0 msgr2=0x7f2140104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:25.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 --2- 172.21.15.6:0/1145957968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100ee0 0x7f2140104eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21300099c0 tx=0x7f2130009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 -- 172.21.15.6:0/1145957968 shutdown_connections 2023-11-17T00:07:25.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 --2- 172.21.15.6:0/1145957968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21401055e0 0x7f214010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 --2- 172.21.15.6:0/1145957968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2140100540 0x7f2140100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 --2- 172.21.15.6:0/1145957968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100ee0 0x7f2140104eb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 -- 172.21.15.6:0/1145957968 >> 172.21.15.6:0/1145957968 conn(0x7f21400fbdc0 msgr2=0x7f21400fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:25.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 -- 172.21.15.6:0/1145957968 shutdown_connections 2023-11-17T00:07:25.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.037+0000 7f2147bef700 1 -- 172.21.15.6:0/1145957968 wait complete. 2023-11-17T00:07:25.403 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:24 smithi186 ceph-mon[131487]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:25.403 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:24 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2143694854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-17T00:07:25.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:24 smithi087 ceph-mon[131182]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:25.403 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:24 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2143694854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-17T00:07:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.038+0000 7f2147bef700 1 Processor -- start 2023-11-17T00:07:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 -- start start 2023-11-17T00:07:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2140100ee0 0x7f2140198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21401055e0 0x7f214019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019d5a0 con 0x7f2140100540 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019d710 con 0x7f21401055e0 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f2147bef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019d8d0 con 0x7f2140100ee0 2023-11-17T00:07:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f214598b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f214598b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38074/0 (socket says 172.21.15.6:38074) 2023-11-17T00:07:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.039+0000 7f214598b700 1 -- 172.21.15.6:0/164394931 learned_addr learned my addr 172.21.15.6:0/164394931 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214618c700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21401055e0 0x7f214019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214518a700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2140100ee0 0x7f2140198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 -- 172.21.15.6:0/164394931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2140100ee0 msgr2=0x7f2140198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2140100ee0 0x7f2140198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 -- 172.21.15.6:0/164394931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21401055e0 msgr2=0x7f214019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21401055e0 0x7f214019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 -- 172.21.15.6:0/164394931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300095f0 con 0x7f2140100540 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.040+0000 7f214598b700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e9a0 tx=0x7f213c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.041+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c009790 con 0x7f2140100540 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.041+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2140110360 con 0x7f2140100540 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.041+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21401101d0 con 0x7f2140100540 2023-11-17T00:07:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.041+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213c010430 con 0x7f2140100540 2023-11-17T00:07:25.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.041+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c018680 con 0x7f2140100540 2023-11-17T00:07:25.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.042+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124005320 con 0x7f2140100540 2023-11-17T00:07:25.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920830 for osd.4 2023-11-17T00:07:25.569 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:25.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.046+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f213c00ca10 con 0x7f2140100540 2023-11-17T00:07:25.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.049+0000 7f2136ffd700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 0x7f212c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:25.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.049+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f213c014070 con 0x7f2140100540 2023-11-17T00:07:25.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.049+0000 7f214518a700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 0x7f212c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.050+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f213c096380 con 0x7f2140100540 2023-11-17T00:07:26.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.050+0000 7f214518a700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 0x7f212c0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fd0 tx=0x7f2130005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:26.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.267+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f2124005cc0 con 0x7f2140100540 2023-11-17T00:07:26.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.267+0000 7f2136ffd700 1 -- 172.21.15.6:0/164394931 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f213c026020 con 0x7f2140100540 2023-11-17T00:07:26.461 INFO:teuthology.orchestra.run.smithi006.stdout:304942678048 2023-11-17T00:07:26.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 msgr2=0x7f212c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 0x7f212c0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fd0 tx=0x7f2130005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 msgr2=0x7f21401985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e9a0 tx=0x7f213c00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 shutdown_connections 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f212c0a1260 0x7f212c0a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2140100540 0x7f21401985e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21401055e0 0x7f214019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 --2- 172.21.15.6:0/164394931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2140100ee0 0x7f2140198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 >> 172.21.15.6:0/164394931 conn(0x7f21400fbdc0 msgr2=0x7f21400fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 shutdown_connections 2023-11-17T00:07:26.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.268+0000 7f2147bef700 1 -- 172.21.15.6:0/164394931 wait complete. 2023-11-17T00:07:26.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:25 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/164394931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-17T00:07:26.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:25 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4244485900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-17T00:07:26.467 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:25 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/164394931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-17T00:07:26.467 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:25 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4244485900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-17T00:07:26.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/1023547901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 msgr2=0x7f7d540ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/1023547901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d540ff2b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009990 tx=0x7f7d50009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/1023547901 shutdown_connections 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/1023547901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d540ff2b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/1023547901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 0x7f7d5410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/1023547901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 0x7f7d5410c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/1023547901 >> 172.21.15.6:0/1023547901 conn(0x7f7d540761a0 msgr2=0x7f7d540765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/1023547901 shutdown_connections 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.467+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/1023547901 wait complete. 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.468+0000 7f7d5a9f7700 1 Processor -- start 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 -- start start 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 0x7f7d541989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 0x7f7d5419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:26.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d470 con 0x7f7d540feee0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d5e0 con 0x7f7d540ff7f0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.469+0000 7f7d5a9f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d7a0 con 0x7f7d5410ca90 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d591f4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 0x7f7d541989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38104/0 (socket says 172.21.15.6:38104) 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 -- 172.21.15.6:0/4244485900 learned_addr learned my addr 172.21.15.6:0/4244485900 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d5a1f6700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 0x7f7d5419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 -- 172.21.15.6:0/4244485900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 msgr2=0x7f7d5419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 0x7f7d5419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 -- 172.21.15.6:0/4244485900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 msgr2=0x7f7d541989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 0x7f7d541989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 -- 172.21.15.6:0/4244485900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d500095f0 con 0x7f7d540feee0 2023-11-17T00:07:26.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d599f5700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005910 tx=0x7f7d50005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.470+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5001d070 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.471+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d50022470 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.471+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d541123c0 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.471+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50017650 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.471+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d54112230 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.472+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d54112550 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.474+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7d5000fad0 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.475+0000 7f7d4affd700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 0x7f7d400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.475+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7d500ce200 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.476+0000 7f7d591f4700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 0x7f7d400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.476+0000 7f7d591f4700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 0x7f7d400a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d54199ad0 tx=0x7f7d44009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.479+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7d50097570 con 0x7f7d540feee0 2023-11-17T00:07:26.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.697+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f7d541998b0 con 0x7f7d540feee0 2023-11-17T00:07:26.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.698+0000 7f7d4affd700 1 -- 172.21.15.6:0/4244485900 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d5009ab90 con 0x7f7d540feee0 2023-11-17T00:07:26.472 INFO:teuthology.orchestra.run.smithi006.stdout:283467841572 2023-11-17T00:07:26.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 msgr2=0x7f7d400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 0x7f7d400a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d54199ad0 tx=0x7f7d44009260 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 msgr2=0x7f7d541984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005910 tx=0x7f7d50005740 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 shutdown_connections 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f7d400a11e0 0x7f7d400a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d540feee0 0x7f7d541984b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d5410ca90 0x7f7d5419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 --2- 172.21.15.6:0/4244485900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d540ff7f0 0x7f7d541989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 >> 172.21.15.6:0/4244485900 conn(0x7f7d540761a0 msgr2=0x7f7d540fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 shutdown_connections 2023-11-17T00:07:26.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:25.700+0000 7f7d5a9f7700 1 -- 172.21.15.6:0/4244485900 wait complete. 2023-11-17T00:07:26.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:25 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/164394931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-17T00:07:26.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:25 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4244485900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-17T00:07:27.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678044 got 304942678048 for osd.10 2023-11-17T00:07:27.080 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:27.110 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:26 smithi087 ceph-mon[131182]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 -- 172.21.15.6:0/3281602908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c81055e0 msgr2=0x7f37c810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 --2- 172.21.15.6:0/3281602908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c81055e0 0x7f37c810d160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b0d0 tx=0x7f37c000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 -- 172.21.15.6:0/3281602908 shutdown_connections 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 --2- 172.21.15.6:0/3281602908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c81055e0 0x7f37c810d160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 --2- 172.21.15.6:0/3281602908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100540 0x7f37c8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 --2- 172.21.15.6:0/3281602908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c8100ee0 0x7f37c8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 -- 172.21.15.6:0/3281602908 >> 172.21.15.6:0/3281602908 conn(0x7f37c80fbdc0 msgr2=0x7f37c80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 -- 172.21.15.6:0/3281602908 shutdown_connections 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.827+0000 7f37cd13d700 1 -- 172.21.15.6:0/3281602908 wait complete. 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.828+0000 7f37cd13d700 1 Processor -- start 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.828+0000 7f37cd13d700 1 -- start start 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100ee0 0x7f37c8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c81055e0 0x7f37c819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c819c5d0 con 0x7f37c8100540 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c819c740 con 0x7f37c81055e0 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37cd13d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c819c900 con 0x7f37c8100ee0 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38124/0 (socket says 172.21.15.6:38124) 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 -- 172.21.15.6:0/2275140310 learned_addr learned my addr 172.21.15.6:0/2275140310 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c759e700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c81055e0 0x7f37c819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c659c700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100ee0 0x7f37c8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 -- 172.21.15.6:0/2275140310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100ee0 msgr2=0x7f37c8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100ee0 0x7f37c8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 -- 172.21.15.6:0/2275140310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c81055e0 msgr2=0x7f37c819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c81055e0 0x7f37c819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.829+0000 7f37c6d9d700 1 -- 172.21.15.6:0/2275140310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c0009d00 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37c6d9d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e9a0 tx=0x7f37b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8009790 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b8010430 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8110370 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8018680 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.830+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c81101e0 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.831+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c8110500 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.834+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f37b80105a0 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.835+0000 7f37affff700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 0x7f37b00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.835+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37b8014070 con 0x7f37c8100540 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.835+0000 7f37c659c700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 0x7f37b00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.836+0000 7f37c659c700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 0x7f37b00a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37c8199cf0 tx=0x7f37bc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:27.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:26.837+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f37b80963f0 con 0x7f37c8100540 2023-11-17T00:07:27.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.053+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f37c81a57e0 con 0x7f37c8100540 2023-11-17T00:07:27.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.053+0000 7f37affff700 1 -- 172.21.15.6:0/2275140310 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f37b8026020 con 0x7f37c8100540 2023-11-17T00:07:27.166 INFO:teuthology.orchestra.run.smithi006.stdout:339302416411 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 msgr2=0x7f37b00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 0x7f37b00a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37c8199cf0 tx=0x7f37bc009260 comp rx=0 tx=0).stop 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 msgr2=0x7f37c81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e9a0 tx=0x7f37b800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 shutdown_connections 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f37b00a1260 0x7f37b00a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37c8100540 0x7f37c81986d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37c81055e0 0x7f37c819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 --2- 172.21.15.6:0/2275140310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c8100ee0 0x7f37c8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:27.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 >> 172.21.15.6:0/2275140310 conn(0x7f37c80fbdc0 msgr2=0x7f37c810b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:27.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 shutdown_connections 2023-11-17T00:07:27.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:27.056+0000 7f37cd13d700 1 -- 172.21.15.6:0/2275140310 wait complete. 2023-11-17T00:07:27.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:26 smithi186 ceph-mon[131487]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:27.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:26 smithi006 ceph-mon[97840]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:27.353 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841570 got 283467841572 for osd.9 2023-11-17T00:07:27.353 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:27.913 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416411 for osd.11 2023-11-17T00:07:27.914 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 -- 172.21.15.6:0/3912638426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6081055e0 msgr2=0x7ff60810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 --2- 172.21.15.6:0/3912638426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6081055e0 0x7ff60810d160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff60000b0d0 tx=0x7ff60000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 -- 172.21.15.6:0/3912638426 shutdown_connections 2023-11-17T00:07:28.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 --2- 172.21.15.6:0/3912638426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6081055e0 0x7ff60810d160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 --2- 172.21.15.6:0/3912638426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff608100540 0x7ff608100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 --2- 172.21.15.6:0/3912638426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100ee0 0x7ff608104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 -- 172.21.15.6:0/3912638426 >> 172.21.15.6:0/3912638426 conn(0x7ff6080fbdc0 msgr2=0x7ff6080fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 -- 172.21.15.6:0/3912638426 shutdown_connections 2023-11-17T00:07:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.062+0000 7ff60f244700 1 -- 172.21.15.6:0/3912638426 wait complete. 2023-11-17T00:07:28.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.063+0000 7ff60f244700 1 Processor -- start 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 -- start start 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff608100ee0 0x7ff60819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6081055e0 0x7ff6081985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608198c30 con 0x7ff608100540 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608198da0 con 0x7ff6081055e0 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60f244700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608198f60 con 0x7ff608100ee0 2023-11-17T00:07:28.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60cfe0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60cfe0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38144/0 (socket says 172.21.15.6:38144) 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60cfe0700 1 -- 172.21.15.6:0/1759887005 learned_addr learned my addr 172.21.15.6:0/1759887005 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff607fff700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff608100ee0 0x7ff60819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.064+0000 7ff60d7e1700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6081055e0 0x7ff6081985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 -- 172.21.15.6:0/1759887005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff608100ee0 msgr2=0x7ff60819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff608100ee0 0x7ff60819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 -- 172.21.15.6:0/1759887005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6081055e0 msgr2=0x7ff6081985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:28.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6081055e0 0x7ff6081985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 -- 172.21.15.6:0/1759887005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009520 con 0x7ff608100540 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60cfe0700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e9a0 tx=0x7ff5fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc009790 con 0x7ff608100540 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5fc010430 con 0x7ff608100540 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.065+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600009d00 con 0x7ff608100540 2023-11-17T00:07:28.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.066+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc018680 con 0x7ff608100540 2023-11-17T00:07:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.066+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff608110370 con 0x7ff608100540 2023-11-17T00:07:28.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.067+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6081101e0 con 0x7ff608100540 2023-11-17T00:07:28.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.070+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff5fc0105a0 con 0x7ff608100540 2023-11-17T00:07:28.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.072+0000 7ff605ffb700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 0x7ff5f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:28.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.072+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff5fc014070 con 0x7ff608100540 2023-11-17T00:07:28.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.072+0000 7ff607fff700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 0x7ff5f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:28.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.073+0000 7ff607fff700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 0x7ff5f80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff608199da0 tx=0x7ff5f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:28.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.076+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff5fc051730 con 0x7ff608100540 2023-11-17T00:07:28.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:27 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2275140310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-17T00:07:28.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:27 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:27 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:27 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2275140310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-17T00:07:28.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:27 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:27 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.239 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:27 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2275140310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-17T00:07:28.239 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:27 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.239 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:27 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:28.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.300+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff608110500 con 0x7ff608100540 2023-11-17T00:07:28.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.300+0000 7ff605ffb700 1 -- 172.21.15.6:0/1759887005 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff5fc0d0030 con 0x7ff608100540 2023-11-17T00:07:28.303 INFO:teuthology.orchestra.run.smithi006.stdout:77309411402 2023-11-17T00:07:28.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 msgr2=0x7ff5f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:28.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 0x7ff5f80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff608199da0 tx=0x7ff5f4009260 comp rx=0 tx=0).stop 2023-11-17T00:07:28.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 msgr2=0x7ff60819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e9a0 tx=0x7ff5fc00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 shutdown_connections 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff5f80a11e0 0x7ff5f80a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff608100540 0x7ff60819e550 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6081055e0 0x7ff6081985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 --2- 172.21.15.6:0/1759887005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff608100ee0 0x7ff60819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 >> 172.21.15.6:0/1759887005 conn(0x7ff6080fbdc0 msgr2=0x7ff60810b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 shutdown_connections 2023-11-17T00:07:28.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:28.303+0000 7ff60f244700 1 -- 172.21.15.6:0/1759887005 wait complete. 2023-11-17T00:07:29.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411402 for osd.2 2023-11-17T00:07:29.019 DEBUG:teuthology.parallel:result is None 2023-11-17T00:07:29.020 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-17T00:07:29.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-17T00:07:29.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:28 smithi006 ceph-mon[97840]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:29.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:28 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1759887005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-17T00:07:29.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:07:29.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:07:29.098 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:29.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:28 smithi087 ceph-mon[131182]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:28 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1759887005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-17T00:07:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:07:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:07:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:29.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:28 smithi186 ceph-mon[131487]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:29.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:28 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1759887005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-17T00:07:29.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:07:29.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:07:29.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:07:29.900 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:29 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:29 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:07:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:29 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:29 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:07:30.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:29 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:29 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:07:30.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:29 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:29 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:07:30.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:29 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:29 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:07:30.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:29 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:07:30.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:29 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:07:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:30 smithi006 ceph-mon[97840]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:30 smithi186 ceph-mon[131487]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:31.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:30 smithi087 ceph-mon[131182]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 -- 172.21.15.6:0/3604045864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc024100ee0 msgr2=0x7fc024104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 --2- 172.21.15.6:0/3604045864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc024100ee0 0x7fc024104eb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0180099c0 tx=0x7fc018009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 -- 172.21.15.6:0/3604045864 shutdown_connections 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 --2- 172.21.15.6:0/3604045864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0241055e0 0x7fc02410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 --2- 172.21.15.6:0/3604045864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100540 0x7fc024100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 --2- 172.21.15.6:0/3604045864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc024100ee0 0x7fc024104eb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 -- 172.21.15.6:0/3604045864 >> 172.21.15.6:0/3604045864 conn(0x7fc0240fbdc0 msgr2=0x7fc0240fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 -- 172.21.15.6:0/3604045864 shutdown_connections 2023-11-17T00:07:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.802+0000 7fc029ea2700 1 -- 172.21.15.6:0/3604045864 wait complete. 2023-11-17T00:07:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.804+0000 7fc029ea2700 1 Processor -- start 2023-11-17T00:07:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.804+0000 7fc029ea2700 1 -- start start 2023-11-17T00:07:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024100540 0x7fc0241985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 0x7fc024198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 0x7fc02419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02419d580 con 0x7fc0241055e0 2023-11-17T00:07:31.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02419d6f0 con 0x7fc024100540 2023-11-17T00:07:31.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc029ea2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02419d8b0 con 0x7fc024100ee0 2023-11-17T00:07:31.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc023fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 0x7fc02419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:31.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc022ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 0x7fc024198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024100540 0x7fc0241985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc023fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 0x7fc02419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:47352/0 (socket says 172.21.15.6:47352) 2023-11-17T00:07:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.805+0000 7fc023fff700 1 -- 172.21.15.6:0/4164651703 learned_addr learned my addr 172.21.15.6:0/4164651703 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 -- 172.21.15.6:0/4164651703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024100540 msgr2=0x7fc0241985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024100540 0x7fc0241985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 -- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 msgr2=0x7fc02419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:31.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 0x7fc02419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:31.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 -- 172.21.15.6:0/4164651703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180095f0 con 0x7fc024100ee0 2023-11-17T00:07:31.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc022ffd700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 0x7fc024198b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b580 tx=0x7fc01800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:31.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01801d070 con 0x7fc024100ee0 2023-11-17T00:07:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc024110360 con 0x7fc024100ee0 2023-11-17T00:07:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc018005050 con 0x7fc024100ee0 2023-11-17T00:07:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018004a00 con 0x7fc024100ee0 2023-11-17T00:07:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.806+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0241101d0 con 0x7fc024100ee0 2023-11-17T00:07:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.808+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0241104f0 con 0x7fc024100ee0 2023-11-17T00:07:31.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.810+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc018005470 con 0x7fc024100ee0 2023-11-17T00:07:31.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.811+0000 7fc020ff9700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 0x7fc00c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:31.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.811+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc0180cdca0 con 0x7fc024100ee0 2023-11-17T00:07:31.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.811+0000 7fc0237fe700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 0x7fc00c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:31.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.812+0000 7fc0237fe700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 0x7fc00c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc014007be0 tx=0x7fc0140073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:31.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:31.813+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc018096fb0 con 0x7fc024100ee0 2023-11-17T00:07:32.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.016+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc02419de10 con 0x7fc00c0a11e0 2023-11-17T00:07:32.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.030+0000 7fc020ff9700 1 -- 172.21.15.6:0/4164651703 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426763 (secure 0 0 0) 0x7fc02419de10 con 0x7fc00c0a11e0 2023-11-17T00:07:32.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:32.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 msgr2=0x7fc00c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:32.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 0x7fc00c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc014007be0 tx=0x7fc0140073d0 comp rx=0 tx=0).stop 2023-11-17T00:07:32.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 msgr2=0x7fc024198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:32.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 0x7fc024198b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b580 tx=0x7fc01800f690 comp rx=0 tx=0).stop 2023-11-17T00:07:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 shutdown_connections 2023-11-17T00:07:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc00c0a11e0 0x7fc00c0a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc0241055e0 0x7fc02419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc024100540 0x7fc0241985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 --2- 172.21.15.6:0/4164651703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc024100ee0 0x7fc024198b00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 >> 172.21.15.6:0/4164651703 conn(0x7fc0240fbdc0 msgr2=0x7fc02410b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 shutdown_connections 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:32.035+0000 7fc029ea2700 1 -- 172.21.15.6:0/4164651703 wait complete. 2023-11-17T00:07:32.062 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:07:32.704 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-11-17T00:07:31.760037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652834,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":253,"num_read_kb":240,"num_write":255,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80752,"kb_used_data":11856,"kb_used_omap":0,"kb_used_meta":68800,"kb_avail":1124861072,"statfs":{"total":1151940427776,"available":1151857737728,"internally_reserved":0,"allocated":12140544,"data_stored":7870278,"data_compressed":318714,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70451200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003362"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826846+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:07:02.826846+0000","last_peered":"2023-11-17T00:07:02.826846+0000","last_clean":"2023-11-17T00:07:02.826846+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:07:02.826846+0000","last_undegraded":"2023-11-17T00:07:02.826846+0000","last_fullsized":"2023-11-17T00:07:02.826846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:29.635832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826015+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:07:02.826015+0000","last_peered":"2023-11-17T00:07:02.826015+0000","last_clean":"2023-11-17T00:07:02.826015+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:07:02.826015+0000","last_undegraded":"2023-11-17T00:07:02.826015+0000","last_fullsized":"2023-11-17T00:07:02.826015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:40.711215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825749+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:07:02.825749+0000","last_peered":"2023-11-17T00:07:02.825749+0000","last_clean":"2023-11-17T00:07:02.825749+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:07:02.825749+0000","last_undegraded":"2023-11-17T00:07:02.825749+0000","last_fullsized":"2023-11-17T00:07:02.825749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:28.698003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905999+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:07:02.905999+0000","last_peered":"2023-11-17T00:07:02.905999+0000","last_clean":"2023-11-17T00:07:02.905999+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:07:02.905999+0000","last_undegraded":"2023-11-17T00:07:02.905999+0000","last_fullsized":"2023-11-17T00:07:02.905999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:48:34.307597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826002+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:07:02.826002+0000","last_peered":"2023-11-17T00:07:02.826002+0000","last_clean":"2023-11-17T00:07:02.826002+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:07:02.826002+0000","last_undegraded":"2023-11-17T00:07:02.826002+0000","last_fullsized":"2023-11-17T00:07:02.826002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:52:25.271013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834121+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:07:02.834121+0000","last_peered":"2023-11-17T00:07:02.834121+0000","last_clean":"2023-11-17T00:07:02.834121+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:07:02.834121+0000","last_undegraded":"2023-11-17T00:07:02.834121+0000","last_fullsized":"2023-11-17T00:07:02.834121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:16:58.736045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822474+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:07:02.822474+0000","last_peered":"2023-11-17T00:07:02.822474+0000","last_clean":"2023-11-17T00:07:02.822474+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:07:02.822474+0000","last_undegraded":"2023-11-17T00:07:02.822474+0000","last_fullsized":"2023-11-17T00:07:02.822474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:52:21.436852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827042+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:07:02.827042+0000","last_peered":"2023-11-17T00:07:02.827042+0000","last_clean":"2023-11-17T00:07:02.827042+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:07:02.827042+0000","last_undegraded":"2023-11-17T00:07:02.827042+0000","last_fullsized":"2023-11-17T00:07:02.827042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:31:42.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905836+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:07:02.905836+0000","last_peered":"2023-11-17T00:07:02.905836+0000","last_clean":"2023-11-17T00:07:02.905836+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:07:02.905836+0000","last_undegraded":"2023-11-17T00:07:02.905836+0000","last_fullsized":"2023-11-17T00:07:02.905836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:31:48.399844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834537+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:07:02.834537+0000","last_peered":"2023-11-17T00:07:02.834537+0000","last_clean":"2023-11-17T00:07:02.834537+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:07:02.834537+0000","last_undegraded":"2023-11-17T00:07:02.834537+0000","last_fullsized":"2023-11-17T00:07:02.834537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:41.253445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823197+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:07:02.823197+0000","last_peered":"2023-11-17T00:07:02.823197+0000","last_clean":"2023-11-17T00:07:02.823197+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:07:02.823197+0000","last_undegraded":"2023-11-17T00:07:02.823197+0000","last_fullsized":"2023-11-17T00:07:02.823197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:18:43.612147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905815+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:07:02.905815+0000","last_peered":"2023-11-17T00:07:02.905815+0000","last_clean":"2023-11-17T00:07:02.905815+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:07:02.905815+0000","last_undegraded":"2023-11-17T00:07:02.905815+0000","last_fullsized":"2023-11-17T00:07:02.905815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:52:55.579056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822297+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:07:02.822297+0000","last_peered":"2023-11-17T00:07:02.822297+0000","last_clean":"2023-11-17T00:07:02.822297+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:07:02.822297+0000","last_undegraded":"2023-11-17T00:07:02.822297+0000","last_fullsized":"2023-11-17T00:07:02.822297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:04:17.073703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825103+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:07:02.825103+0000","last_peered":"2023-11-17T00:07:02.825103+0000","last_clean":"2023-11-17T00:07:02.825103+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:07:02.825103+0000","last_undegraded":"2023-11-17T00:07:02.825103+0000","last_fullsized":"2023-11-17T00:07:02.825103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:28:42.750181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825495+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:07:02.825495+0000","last_peered":"2023-11-17T00:07:02.825495+0000","last_clean":"2023-11-17T00:07:02.825495+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:07:02.825495+0000","last_undegraded":"2023-11-17T00:07:02.825495+0000","last_fullsized":"2023-11-17T00:07:02.825495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:36:00.587367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904968+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:07:02.904968+0000","last_peered":"2023-11-17T00:07:02.904968+0000","last_clean":"2023-11-17T00:07:02.904968+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:07:02.904968+0000","last_undegraded":"2023-11-17T00:07:02.904968+0000","last_fullsized":"2023-11-17T00:07:02.904968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:15.663690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826706+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:07:02.826706+0000","last_peered":"2023-11-17T00:07:02.826706+0000","last_clean":"2023-11-17T00:07:02.826706+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:07:02.826706+0000","last_undegraded":"2023-11-17T00:07:02.826706+0000","last_fullsized":"2023-11-17T00:07:02.826706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:07:48.794377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904844+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:07:02.904844+0000","last_peered":"2023-11-17T00:07:02.904844+0000","last_clean":"2023-11-17T00:07:02.904844+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:07:02.904844+0000","last_undegraded":"2023-11-17T00:07:02.904844+0000","last_fullsized":"2023-11-17T00:07:02.904844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:48.005898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827922+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:07:02.827922+0000","last_peered":"2023-11-17T00:07:02.827922+0000","last_clean":"2023-11-17T00:07:02.827922+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:07:02.827922+0000","last_undegraded":"2023-11-17T00:07:02.827922+0000","last_fullsized":"2023-11-17T00:07:02.827922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:06:26.414630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826839+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:07:02.826839+0000","last_peered":"2023-11-17T00:07:02.826839+0000","last_clean":"2023-11-17T00:07:02.826839+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:07:02.826839+0000","last_undegraded":"2023-11-17T00:07:02.826839+0000","last_fullsized":"2023-11-17T00:07:02.826839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:36:59.477340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904660+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:07:02.904660+0000","last_peered":"2023-11-17T00:07:02.904660+0000","last_clean":"2023-11-17T00:07:02.904660+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:07:02.904660+0000","last_undegraded":"2023-11-17T00:07:02.904660+0000","last_fullsized":"2023-11-17T00:07:02.904660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:25.085518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827626+0000","last_change":"2023-11-17T00:06:12.628519+0000","last_active":"2023-11-17T00:07:02.827626+0000","last_peered":"2023-11-17T00:07:02.827626+0000","last_clean":"2023-11-17T00:07:02.827626+0000","last_became_active":"2023-11-17T00:06:12.627360+0000","last_became_peered":"2023-11-17T00:06:12.627360+0000","last_unstale":"2023-11-17T00:07:02.827626+0000","last_undegraded":"2023-11-17T00:07:02.827626+0000","last_fullsized":"2023-11-17T00:07:02.827626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:09:51.885082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904221+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:07:02.904221+0000","last_peered":"2023-11-17T00:07:02.904221+0000","last_clean":"2023-11-17T00:07:02.904221+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:07:02.904221+0000","last_undegraded":"2023-11-17T00:07:02.904221+0000","last_fullsized":"2023-11-17T00:07:02.904221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:51:29.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827768+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:07:02.827768+0000","last_peered":"2023-11-17T00:07:02.827768+0000","last_clean":"2023-11-17T00:07:02.827768+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:07:02.827768+0000","last_undegraded":"2023-11-17T00:07:02.827768+0000","last_fullsized":"2023-11-17T00:07:02.827768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:23.232685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826329+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:07:02.826329+0000","last_peered":"2023-11-17T00:07:02.826329+0000","last_clean":"2023-11-17T00:07:02.826329+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:07:02.826329+0000","last_undegraded":"2023-11-17T00:07:02.826329+0000","last_fullsized":"2023-11-17T00:07:02.826329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905006+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:07:02.905006+0000","last_peered":"2023-11-17T00:07:02.905006+0000","last_clean":"2023-11-17T00:07:02.905006+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:07:02.905006+0000","last_undegraded":"2023-11-17T00:07:02.905006+0000","last_fullsized":"2023-11-17T00:07:02.905006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:14:06.574179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826705+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:07:02.826705+0000","last_peered":"2023-11-17T00:07:02.826705+0000","last_clean":"2023-11-17T00:07:02.826705+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:07:02.826705+0000","last_undegraded":"2023-11-17T00:07:02.826705+0000","last_fullsized":"2023-11-17T00:07:02.826705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:02:22.741567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904169+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:07:02.904169+0000","last_peered":"2023-11-17T00:07:02.904169+0000","last_clean":"2023-11-17T00:07:02.904169+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:07:02.904169+0000","last_undegraded":"2023-11-17T00:07:02.904169+0000","last_fullsized":"2023-11-17T00:07:02.904169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:31:07.179458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904738+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:07:02.904738+0000","last_peered":"2023-11-17T00:07:02.904738+0000","last_clean":"2023-11-17T00:07:02.904738+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:07:02.904738+0000","last_undegraded":"2023-11-17T00:07:02.904738+0000","last_fullsized":"2023-11-17T00:07:02.904738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:39:24.141254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825793+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:07:02.825793+0000","last_peered":"2023-11-17T00:07:02.825793+0000","last_clean":"2023-11-17T00:07:02.825793+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:07:02.825793+0000","last_undegraded":"2023-11-17T00:07:02.825793+0000","last_fullsized":"2023-11-17T00:07:02.825793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:29:35.519337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823032+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:07:02.823032+0000","last_peered":"2023-11-17T00:07:02.823032+0000","last_clean":"2023-11-17T00:07:02.823032+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:07:02.823032+0000","last_undegraded":"2023-11-17T00:07:02.823032+0000","last_fullsized":"2023-11-17T00:07:02.823032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:04.449613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827448+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:07:02.827448+0000","last_peered":"2023-11-17T00:07:02.827448+0000","last_clean":"2023-11-17T00:07:02.827448+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:07:02.827448+0000","last_undegraded":"2023-11-17T00:07:02.827448+0000","last_fullsized":"2023-11-17T00:07:02.827448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:42:04.048485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823255+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:07:02.823255+0000","last_peered":"2023-11-17T00:07:02.823255+0000","last_clean":"2023-11-17T00:07:02.823255+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:07:02.823255+0000","last_undegraded":"2023-11-17T00:07:02.823255+0000","last_fullsized":"2023-11-17T00:07:02.823255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:55:03.575265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827611+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:07:02.827611+0000","last_peered":"2023-11-17T00:07:02.827611+0000","last_clean":"2023-11-17T00:07:02.827611+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:07:02.827611+0000","last_undegraded":"2023-11-17T00:07:02.827611+0000","last_fullsized":"2023-11-17T00:07:02.827611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:25:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827552+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:07:02.827552+0000","last_peered":"2023-11-17T00:07:02.827552+0000","last_clean":"2023-11-17T00:07:02.827552+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:07:02.827552+0000","last_undegraded":"2023-11-17T00:07:02.827552+0000","last_fullsized":"2023-11-17T00:07:02.827552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:54:56.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834106+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:07:02.834106+0000","last_peered":"2023-11-17T00:07:02.834106+0000","last_clean":"2023-11-17T00:07:02.834106+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:07:02.834106+0000","last_undegraded":"2023-11-17T00:07:02.834106+0000","last_fullsized":"2023-11-17T00:07:02.834106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:36:35.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.824231+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:07:02.824231+0000","last_peered":"2023-11-17T00:07:02.824231+0000","last_clean":"2023-11-17T00:07:02.824231+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:07:02.824231+0000","last_undegraded":"2023-11-17T00:07:02.824231+0000","last_fullsized":"2023-11-17T00:07:02.824231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:31:30.495283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.907032+0000","last_change":"2023-11-17T00:07:02.907032+0000","last_active":"2023-11-17T00:07:02.907032+0000","last_peered":"2023-11-17T00:07:02.907032+0000","last_clean":"2023-11-17T00:07:02.907032+0000","last_became_active":"2023-11-17T00:07:02.906576+0000","last_became_peered":"2023-11-17T00:07:02.906576+0000","last_unstale":"2023-11-17T00:07:02.907032+0000","last_undegraded":"2023-11-17T00:07:02.907032+0000","last_fullsized":"2023-11-17T00:07:02.907032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:17:37.002477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821900+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:07:02.821900+0000","last_peered":"2023-11-17T00:07:02.821900+0000","last_clean":"2023-11-17T00:07:02.821900+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:07:02.821900+0000","last_undegraded":"2023-11-17T00:07:02.821900+0000","last_fullsized":"2023-11-17T00:07:02.821900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:25:54.909513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826999+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:07:02.826999+0000","last_peered":"2023-11-17T00:07:02.826999+0000","last_clean":"2023-11-17T00:07:02.826999+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:07:02.826999+0000","last_undegraded":"2023-11-17T00:07:02.826999+0000","last_fullsized":"2023-11-17T00:07:02.826999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:23:16.172360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826053+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:07:02.826053+0000","last_peered":"2023-11-17T00:07:02.826053+0000","last_clean":"2023-11-17T00:07:02.826053+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:07:02.826053+0000","last_undegraded":"2023-11-17T00:07:02.826053+0000","last_fullsized":"2023-11-17T00:07:02.826053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:41.604524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822579+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:07:02.822579+0000","last_peered":"2023-11-17T00:07:02.822579+0000","last_clean":"2023-11-17T00:07:02.822579+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:07:02.822579+0000","last_undegraded":"2023-11-17T00:07:02.822579+0000","last_fullsized":"2023-11-17T00:07:02.822579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:33:24.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903950+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:07:02.903950+0000","last_peered":"2023-11-17T00:07:02.903950+0000","last_clean":"2023-11-17T00:07:02.903950+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:07:02.903950+0000","last_undegraded":"2023-11-17T00:07:02.903950+0000","last_fullsized":"2023-11-17T00:07:02.903950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:26.168940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826025+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:07:02.826025+0000","last_peered":"2023-11-17T00:07:02.826025+0000","last_clean":"2023-11-17T00:07:02.826025+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:07:02.826025+0000","last_undegraded":"2023-11-17T00:07:02.826025+0000","last_fullsized":"2023-11-17T00:07:02.826025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:22:22.019983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834455+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:07:02.834455+0000","last_peered":"2023-11-17T00:07:02.834455+0000","last_clean":"2023-11-17T00:07:02.834455+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:07:02.834455+0000","last_undegraded":"2023-11-17T00:07:02.834455+0000","last_fullsized":"2023-11-17T00:07:02.834455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:44:14.351304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905159+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:07:02.905159+0000","last_peered":"2023-11-17T00:07:02.905159+0000","last_clean":"2023-11-17T00:07:02.905159+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:07:02.905159+0000","last_undegraded":"2023-11-17T00:07:02.905159+0000","last_fullsized":"2023-11-17T00:07:02.905159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:24.835195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903836+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:07:02.903836+0000","last_peered":"2023-11-17T00:07:02.903836+0000","last_clean":"2023-11-17T00:07:02.903836+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:07:02.903836+0000","last_undegraded":"2023-11-17T00:07:02.903836+0000","last_fullsized":"2023-11-17T00:07:02.903836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:53:26.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823087+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:07:02.823087+0000","last_peered":"2023-11-17T00:07:02.823087+0000","last_clean":"2023-11-17T00:07:02.823087+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:07:02.823087+0000","last_undegraded":"2023-11-17T00:07:02.823087+0000","last_fullsized":"2023-11-17T00:07:02.823087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:33:35.310318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826437+0000","last_change":"2023-11-17T00:06:12.638172+0000","last_active":"2023-11-17T00:07:02.826437+0000","last_peered":"2023-11-17T00:07:02.826437+0000","last_clean":"2023-11-17T00:07:02.826437+0000","last_became_active":"2023-11-17T00:06:12.637227+0000","last_became_peered":"2023-11-17T00:06:12.637227+0000","last_unstale":"2023-11-17T00:07:02.826437+0000","last_undegraded":"2023-11-17T00:07:02.826437+0000","last_fullsized":"2023-11-17T00:07:02.826437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:48:57.804467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905310+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:07:02.905310+0000","last_peered":"2023-11-17T00:07:02.905310+0000","last_clean":"2023-11-17T00:07:02.905310+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:07:02.905310+0000","last_undegraded":"2023-11-17T00:07:02.905310+0000","last_fullsized":"2023-11-17T00:07:02.905310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:13.073808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822885+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:07:02.822885+0000","last_peered":"2023-11-17T00:07:02.822885+0000","last_clean":"2023-11-17T00:07:02.822885+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:07:02.822885+0000","last_undegraded":"2023-11-17T00:07:02.822885+0000","last_fullsized":"2023-11-17T00:07:02.822885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:56:49.301970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904320+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:07:02.904320+0000","last_peered":"2023-11-17T00:07:02.904320+0000","last_clean":"2023-11-17T00:07:02.904320+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:07:02.904320+0000","last_undegraded":"2023-11-17T00:07:02.904320+0000","last_fullsized":"2023-11-17T00:07:02.904320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:26.693598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903985+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:07:02.903985+0000","last_peered":"2023-11-17T00:07:02.903985+0000","last_clean":"2023-11-17T00:07:02.903985+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:07:02.903985+0000","last_undegraded":"2023-11-17T00:07:02.903985+0000","last_fullsized":"2023-11-17T00:07:02.903985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:25.639542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":166,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:27.923944+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:07:27.923944+0000","last_peered":"2023-11-17T00:07:27.923944+0000","last_clean":"2023-11-17T00:07:27.923944+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:07:27.923944+0000","last_undegraded":"2023-11-17T00:07:27.923944+0000","last_fullsized":"2023-11-17T00:07:27.923944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:16.841523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61422,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":115,"num_write":29,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826378+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:07:02.826378+0000","last_peered":"2023-11-17T00:07:02.826378+0000","last_clean":"2023-11-17T00:07:02.826378+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:07:02.826378+0000","last_undegraded":"2023-11-17T00:07:02.826378+0000","last_fullsized":"2023-11-17T00:07:02.826378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:28:55.207231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823519+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:07:02.823519+0000","last_peered":"2023-11-17T00:07:02.823519+0000","last_clean":"2023-11-17T00:07:02.823519+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:07:02.823519+0000","last_undegraded":"2023-11-17T00:07:02.823519+0000","last_fullsized":"2023-11-17T00:07:02.823519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:36:08.220274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:03.429143+0000","last_change":"2023-11-17T00:07:03.428692+0000","last_active":"2023-11-17T00:07:03.429143+0000","last_peered":"2023-11-17T00:07:03.429143+0000","last_clean":"2023-11-17T00:07:03.429143+0000","last_became_active":"2023-11-17T00:07:02.832461+0000","last_became_peered":"2023-11-17T00:07:02.832461+0000","last_unstale":"2023-11-17T00:07:03.429143+0000","last_undegraded":"2023-11-17T00:07:03.429143+0000","last_fullsized":"2023-11-17T00:07:03.429143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:24.423317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822030+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:07:02.822030+0000","last_peered":"2023-11-17T00:07:02.822030+0000","last_clean":"2023-11-17T00:07:02.822030+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:07:02.822030+0000","last_undegraded":"2023-11-17T00:07:02.822030+0000","last_fullsized":"2023-11-17T00:07:02.822030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:11:44.312321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.831797+0000","last_change":"2023-11-17T00:07:02.831797+0000","last_active":"2023-11-17T00:07:02.831797+0000","last_peered":"2023-11-17T00:07:02.831797+0000","last_clean":"2023-11-17T00:07:02.831797+0000","last_became_active":"2023-11-17T00:07:02.831339+0000","last_became_peered":"2023-11-17T00:07:02.831339+0000","last_unstale":"2023-11-17T00:07:02.831797+0000","last_undegraded":"2023-11-17T00:07:02.831797+0000","last_fullsized":"2023-11-17T00:07:02.831797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:58:34.477233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823406+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:07:02.823406+0000","last_peered":"2023-11-17T00:07:02.823406+0000","last_clean":"2023-11-17T00:07:02.823406+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:07:02.823406+0000","last_undegraded":"2023-11-17T00:07:02.823406+0000","last_fullsized":"2023-11-17T00:07:02.823406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:39:41.389091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905543+0000","last_change":"2023-11-17T00:07:02.905543+0000","last_active":"2023-11-17T00:07:02.905543+0000","last_peered":"2023-11-17T00:07:02.905543+0000","last_clean":"2023-11-17T00:07:02.905543+0000","last_became_active":"2023-11-17T00:07:02.905087+0000","last_became_peered":"2023-11-17T00:07:02.905087+0000","last_unstale":"2023-11-17T00:07:02.905543+0000","last_undegraded":"2023-11-17T00:07:02.905543+0000","last_fullsized":"2023-11-17T00:07:02.905543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:40:53.411044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904498+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:07:02.904498+0000","last_peered":"2023-11-17T00:07:02.904498+0000","last_clean":"2023-11-17T00:07:02.904498+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:07:02.904498+0000","last_undegraded":"2023-11-17T00:07:02.904498+0000","last_fullsized":"2023-11-17T00:07:02.904498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:10:09.796313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905570+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:07:02.905570+0000","last_peered":"2023-11-17T00:07:02.905570+0000","last_clean":"2023-11-17T00:07:02.905570+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:07:02.905570+0000","last_undegraded":"2023-11-17T00:07:02.905570+0000","last_fullsized":"2023-11-17T00:07:02.905570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:08.691707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823381+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:07:02.823381+0000","last_peered":"2023-11-17T00:07:02.823381+0000","last_clean":"2023-11-17T00:07:02.823381+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:07:02.823381+0000","last_undegraded":"2023-11-17T00:07:02.823381+0000","last_fullsized":"2023-11-17T00:07:02.823381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:29:14.359493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822441+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:07:02.822441+0000","last_peered":"2023-11-17T00:07:02.822441+0000","last_clean":"2023-11-17T00:07:02.822441+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:07:02.822441+0000","last_undegraded":"2023-11-17T00:07:02.822441+0000","last_fullsized":"2023-11-17T00:07:02.822441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:02.079300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825837+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:07:02.825837+0000","last_peered":"2023-11-17T00:07:02.825837+0000","last_clean":"2023-11-17T00:07:02.825837+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:07:02.825837+0000","last_undegraded":"2023-11-17T00:07:02.825837+0000","last_fullsized":"2023-11-17T00:07:02.825837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:46:21.300209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822789+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:07:02.822789+0000","last_peered":"2023-11-17T00:07:02.822789+0000","last_clean":"2023-11-17T00:07:02.822789+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:07:02.822789+0000","last_undegraded":"2023-11-17T00:07:02.822789+0000","last_fullsized":"2023-11-17T00:07:02.822789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:46:38.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822639+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:07:02.822639+0000","last_peered":"2023-11-17T00:07:02.822639+0000","last_clean":"2023-11-17T00:07:02.822639+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:07:02.822639+0000","last_undegraded":"2023-11-17T00:07:02.822639+0000","last_fullsized":"2023-11-17T00:07:02.822639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:20.570854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826407+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:07:02.826407+0000","last_peered":"2023-11-17T00:07:02.826407+0000","last_clean":"2023-11-17T00:07:02.826407+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:07:02.826407+0000","last_undegraded":"2023-11-17T00:07:02.826407+0000","last_fullsized":"2023-11-17T00:07:02.826407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:19.334144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827339+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:07:02.827339+0000","last_peered":"2023-11-17T00:07:02.827339+0000","last_clean":"2023-11-17T00:07:02.827339+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:07:02.827339+0000","last_undegraded":"2023-11-17T00:07:02.827339+0000","last_fullsized":"2023-11-17T00:07:02.827339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:59.198507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827452+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:07:02.827452+0000","last_peered":"2023-11-17T00:07:02.827452+0000","last_clean":"2023-11-17T00:07:02.827452+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:07:02.827452+0000","last_undegraded":"2023-11-17T00:07:02.827452+0000","last_fullsized":"2023-11-17T00:07:02.827452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:02:41.038095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904302+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:07:02.904302+0000","last_peered":"2023-11-17T00:07:02.904302+0000","last_clean":"2023-11-17T00:07:02.904302+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:07:02.904302+0000","last_undegraded":"2023-11-17T00:07:02.904302+0000","last_fullsized":"2023-11-17T00:07:02.904302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:33:31.825193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834456+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:07:02.834456+0000","last_peered":"2023-11-17T00:07:02.834456+0000","last_clean":"2023-11-17T00:07:02.834456+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:07:02.834456+0000","last_undegraded":"2023-11-17T00:07:02.834456+0000","last_fullsized":"2023-11-17T00:07:02.834456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:07.757713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822319+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:07:02.822319+0000","last_peered":"2023-11-17T00:07:02.822319+0000","last_clean":"2023-11-17T00:07:02.822319+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:07:02.822319+0000","last_undegraded":"2023-11-17T00:07:02.822319+0000","last_fullsized":"2023-11-17T00:07:02.822319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:31.546886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822628+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:07:02.822628+0000","last_peered":"2023-11-17T00:07:02.822628+0000","last_clean":"2023-11-17T00:07:02.822628+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:07:02.822628+0000","last_undegraded":"2023-11-17T00:07:02.822628+0000","last_fullsized":"2023-11-17T00:07:02.822628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:57.597621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834235+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:07:02.834235+0000","last_peered":"2023-11-17T00:07:02.834235+0000","last_clean":"2023-11-17T00:07:02.834235+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:07:02.834235+0000","last_undegraded":"2023-11-17T00:07:02.834235+0000","last_fullsized":"2023-11-17T00:07:02.834235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:35:18.172438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822421+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:07:02.822421+0000","last_peered":"2023-11-17T00:07:02.822421+0000","last_clean":"2023-11-17T00:07:02.822421+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:07:02.822421+0000","last_undegraded":"2023-11-17T00:07:02.822421+0000","last_fullsized":"2023-11-17T00:07:02.822421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:25:55.041206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822811+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:07:02.822811+0000","last_peered":"2023-11-17T00:07:02.822811+0000","last_clean":"2023-11-17T00:07:02.822811+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:07:02.822811+0000","last_undegraded":"2023-11-17T00:07:02.822811+0000","last_fullsized":"2023-11-17T00:07:02.822811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:11:58.695080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821790+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:07:02.821790+0000","last_peered":"2023-11-17T00:07:02.821790+0000","last_clean":"2023-11-17T00:07:02.821790+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:07:02.821790+0000","last_undegraded":"2023-11-17T00:07:02.821790+0000","last_fullsized":"2023-11-17T00:07:02.821790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:19:50.407968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822812+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:07:02.822812+0000","last_peered":"2023-11-17T00:07:02.822812+0000","last_clean":"2023-11-17T00:07:02.822812+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:07:02.822812+0000","last_undegraded":"2023-11-17T00:07:02.822812+0000","last_fullsized":"2023-11-17T00:07:02.822812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:40:47.701107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826430+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:07:02.826430+0000","last_peered":"2023-11-17T00:07:02.826430+0000","last_clean":"2023-11-17T00:07:02.826430+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:07:02.826430+0000","last_undegraded":"2023-11-17T00:07:02.826430+0000","last_fullsized":"2023-11-17T00:07:02.826430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:19.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905156+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:07:02.905156+0000","last_peered":"2023-11-17T00:07:02.905156+0000","last_clean":"2023-11-17T00:07:02.905156+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:07:02.905156+0000","last_undegraded":"2023-11-17T00:07:02.905156+0000","last_fullsized":"2023-11-17T00:07:02.905156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:59:31.001093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822513+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:07:02.822513+0000","last_peered":"2023-11-17T00:07:02.822513+0000","last_clean":"2023-11-17T00:07:02.822513+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:07:02.822513+0000","last_undegraded":"2023-11-17T00:07:02.822513+0000","last_fullsized":"2023-11-17T00:07:02.822513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:02:56.262488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823767+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:07:02.823767+0000","last_peered":"2023-11-17T00:07:02.823767+0000","last_clean":"2023-11-17T00:07:02.823767+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:07:02.823767+0000","last_undegraded":"2023-11-17T00:07:02.823767+0000","last_fullsized":"2023-11-17T00:07:02.823767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:02:15.925740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821770+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:07:02.821770+0000","last_peered":"2023-11-17T00:07:02.821770+0000","last_clean":"2023-11-17T00:07:02.821770+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:07:02.821770+0000","last_undegraded":"2023-11-17T00:07:02.821770+0000","last_fullsized":"2023-11-17T00:07:02.821770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:57:19.724909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906620+0000","last_change":"2023-11-17T00:07:02.906620+0000","last_active":"2023-11-17T00:07:02.906620+0000","last_peered":"2023-11-17T00:07:02.906620+0000","last_clean":"2023-11-17T00:07:02.906620+0000","last_became_active":"2023-11-17T00:07:02.906237+0000","last_became_peered":"2023-11-17T00:07:02.906237+0000","last_unstale":"2023-11-17T00:07:02.906620+0000","last_undegraded":"2023-11-17T00:07:02.906620+0000","last_fullsized":"2023-11-17T00:07:02.906620+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:45:43.859463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827046+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:07:02.827046+0000","last_peered":"2023-11-17T00:07:02.827046+0000","last_clean":"2023-11-17T00:07:02.827046+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:07:02.827046+0000","last_undegraded":"2023-11-17T00:07:02.827046+0000","last_fullsized":"2023-11-17T00:07:02.827046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:45:13.061257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826500+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:07:02.826500+0000","last_peered":"2023-11-17T00:07:02.826500+0000","last_clean":"2023-11-17T00:07:02.826500+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:07:02.826500+0000","last_undegraded":"2023-11-17T00:07:02.826500+0000","last_fullsized":"2023-11-17T00:07:02.826500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:47:36.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822518+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:07:02.822518+0000","last_peered":"2023-11-17T00:07:02.822518+0000","last_clean":"2023-11-17T00:07:02.822518+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:07:02.822518+0000","last_undegraded":"2023-11-17T00:07:02.822518+0000","last_fullsized":"2023-11-17T00:07:02.822518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:28:46.236875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826859+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:07:02.826859+0000","last_peered":"2023-11-17T00:07:02.826859+0000","last_clean":"2023-11-17T00:07:02.826859+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:07:02.826859+0000","last_undegraded":"2023-11-17T00:07:02.826859+0000","last_fullsized":"2023-11-17T00:07:02.826859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:54:22.051875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904721+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:07:02.904721+0000","last_peered":"2023-11-17T00:07:02.904721+0000","last_clean":"2023-11-17T00:07:02.904721+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:07:02.904721+0000","last_undegraded":"2023-11-17T00:07:02.904721+0000","last_fullsized":"2023-11-17T00:07:02.904721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:47:50.659334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827650+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:07:02.827650+0000","last_peered":"2023-11-17T00:07:02.827650+0000","last_clean":"2023-11-17T00:07:02.827650+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:07:02.827650+0000","last_undegraded":"2023-11-17T00:07:02.827650+0000","last_fullsized":"2023-11-17T00:07:02.827650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:54.546763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826712+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:07:02.826712+0000","last_peered":"2023-11-17T00:07:02.826712+0000","last_clean":"2023-11-17T00:07:02.826712+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:07:02.826712+0000","last_undegraded":"2023-11-17T00:07:02.826712+0000","last_fullsized":"2023-11-17T00:07:02.826712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:53.773020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906453+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:07:02.906453+0000","last_peered":"2023-11-17T00:07:02.906453+0000","last_clean":"2023-11-17T00:07:02.906453+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:07:02.906453+0000","last_undegraded":"2023-11-17T00:07:02.906453+0000","last_fullsized":"2023-11-17T00:07:02.906453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:24:35.297870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826152+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:07:02.826152+0000","last_peered":"2023-11-17T00:07:02.826152+0000","last_clean":"2023-11-17T00:07:02.826152+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:07:02.826152+0000","last_undegraded":"2023-11-17T00:07:02.826152+0000","last_fullsized":"2023-11-17T00:07:02.826152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:58.836590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906353+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:07:02.906353+0000","last_peered":"2023-11-17T00:07:02.906353+0000","last_clean":"2023-11-17T00:07:02.906353+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:07:02.906353+0000","last_undegraded":"2023-11-17T00:07:02.906353+0000","last_fullsized":"2023-11-17T00:07:02.906353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:24:07.960782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905119+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:07:02.905119+0000","last_peered":"2023-11-17T00:07:02.905119+0000","last_clean":"2023-11-17T00:07:02.905119+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:07:02.905119+0000","last_undegraded":"2023-11-17T00:07:02.905119+0000","last_fullsized":"2023-11-17T00:07:02.905119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:18:12.956129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825020+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:07:02.825020+0000","last_peered":"2023-11-17T00:07:02.825020+0000","last_clean":"2023-11-17T00:07:02.825020+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:07:02.825020+0000","last_undegraded":"2023-11-17T00:07:02.825020+0000","last_fullsized":"2023-11-17T00:07:02.825020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:41:11.727361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904629+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:07:02.904629+0000","last_peered":"2023-11-17T00:07:02.904629+0000","last_clean":"2023-11-17T00:07:02.904629+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:07:02.904629+0000","last_undegraded":"2023-11-17T00:07:02.904629+0000","last_fullsized":"2023-11-17T00:07:02.904629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:48:14.535390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825694+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:07:02.825694+0000","last_peered":"2023-11-17T00:07:02.825694+0000","last_clean":"2023-11-17T00:07:02.825694+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:07:02.825694+0000","last_undegraded":"2023-11-17T00:07:02.825694+0000","last_fullsized":"2023-11-17T00:07:02.825694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:43:12.873004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825643+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:07:02.825643+0000","last_peered":"2023-11-17T00:07:02.825643+0000","last_clean":"2023-11-17T00:07:02.825643+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:07:02.825643+0000","last_undegraded":"2023-11-17T00:07:02.825643+0000","last_fullsized":"2023-11-17T00:07:02.825643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:13:29.632520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905501+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:07:02.905501+0000","last_peered":"2023-11-17T00:07:02.905501+0000","last_clean":"2023-11-17T00:07:02.905501+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:07:02.905501+0000","last_undegraded":"2023-11-17T00:07:02.905501+0000","last_fullsized":"2023-11-17T00:07:02.905501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:31:46.721192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.832991+0000","last_change":"2023-11-17T00:07:02.831541+0000","last_active":"2023-11-17T00:07:02.832991+0000","last_peered":"2023-11-17T00:07:02.832991+0000","last_clean":"2023-11-17T00:07:02.832991+0000","last_became_active":"2023-11-17T00:07:02.831030+0000","last_became_peered":"2023-11-17T00:07:02.831030+0000","last_unstale":"2023-11-17T00:07:02.832991+0000","last_undegraded":"2023-11-17T00:07:02.832991+0000","last_fullsized":"2023-11-17T00:07:02.832991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:33:02.117643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827288+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:07:02.827288+0000","last_peered":"2023-11-17T00:07:02.827288+0000","last_clean":"2023-11-17T00:07:02.827288+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:07:02.827288+0000","last_undegraded":"2023-11-17T00:07:02.827288+0000","last_fullsized":"2023-11-17T00:07:02.827288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:07:19.757734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905512+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:07:02.905512+0000","last_peered":"2023-11-17T00:07:02.905512+0000","last_clean":"2023-11-17T00:07:02.905512+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:07:02.905512+0000","last_undegraded":"2023-11-17T00:07:02.905512+0000","last_fullsized":"2023-11-17T00:07:02.905512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:34:05.676105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825096+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:07:02.825096+0000","last_peered":"2023-11-17T00:07:02.825096+0000","last_clean":"2023-11-17T00:07:02.825096+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:07:02.825096+0000","last_undegraded":"2023-11-17T00:07:02.825096+0000","last_fullsized":"2023-11-17T00:07:02.825096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:47.945629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823108+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:07:02.823108+0000","last_peered":"2023-11-17T00:07:02.823108+0000","last_clean":"2023-11-17T00:07:02.823108+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:07:02.823108+0000","last_undegraded":"2023-11-17T00:07:02.823108+0000","last_fullsized":"2023-11-17T00:07:02.823108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:30.496908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904122+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:07:02.904122+0000","last_peered":"2023-11-17T00:07:02.904122+0000","last_clean":"2023-11-17T00:07:02.904122+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:07:02.904122+0000","last_undegraded":"2023-11-17T00:07:02.904122+0000","last_fullsized":"2023-11-17T00:07:02.904122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:10:06.214442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827858+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:07:02.827858+0000","last_peered":"2023-11-17T00:07:02.827858+0000","last_clean":"2023-11-17T00:07:02.827858+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:07:02.827858+0000","last_undegraded":"2023-11-17T00:07:02.827858+0000","last_fullsized":"2023-11-17T00:07:02.827858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:44:49.714370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823144+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:07:02.823144+0000","last_peered":"2023-11-17T00:07:02.823144+0000","last_clean":"2023-11-17T00:07:02.823144+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:07:02.823144+0000","last_undegraded":"2023-11-17T00:07:02.823144+0000","last_fullsized":"2023-11-17T00:07:02.823144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:43:01.046315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906790+0000","last_change":"2023-11-17T00:07:02.906790+0000","last_active":"2023-11-17T00:07:02.906790+0000","last_peered":"2023-11-17T00:07:02.906790+0000","last_clean":"2023-11-17T00:07:02.906790+0000","last_became_active":"2023-11-17T00:07:02.906439+0000","last_became_peered":"2023-11-17T00:07:02.906439+0000","last_unstale":"2023-11-17T00:07:02.906790+0000","last_undegraded":"2023-11-17T00:07:02.906790+0000","last_fullsized":"2023-11-17T00:07:02.906790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:03.827866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823276+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:07:02.823276+0000","last_peered":"2023-11-17T00:07:02.823276+0000","last_clean":"2023-11-17T00:07:02.823276+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:07:02.823276+0000","last_undegraded":"2023-11-17T00:07:02.823276+0000","last_fullsized":"2023-11-17T00:07:02.823276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:36:34.182907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822997+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:07:02.822997+0000","last_peered":"2023-11-17T00:07:02.822997+0000","last_clean":"2023-11-17T00:07:02.822997+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:07:02.822997+0000","last_undegraded":"2023-11-17T00:07:02.822997+0000","last_fullsized":"2023-11-17T00:07:02.822997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:24:25.194919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906389+0000","last_change":"2023-11-17T00:07:02.906389+0000","last_active":"2023-11-17T00:07:02.906389+0000","last_peered":"2023-11-17T00:07:02.906389+0000","last_clean":"2023-11-17T00:07:02.906389+0000","last_became_active":"2023-11-17T00:07:02.905885+0000","last_became_peered":"2023-11-17T00:07:02.905885+0000","last_unstale":"2023-11-17T00:07:02.906389+0000","last_undegraded":"2023-11-17T00:07:02.906389+0000","last_fullsized":"2023-11-17T00:07:02.906389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:01:26.319051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:03.431743+0000","last_change":"2023-11-17T00:07:03.431743+0000","last_active":"2023-11-17T00:07:03.431743+0000","last_peered":"2023-11-17T00:07:03.431743+0000","last_clean":"2023-11-17T00:07:03.431743+0000","last_became_active":"2023-11-17T00:07:02.907841+0000","last_became_peered":"2023-11-17T00:07:02.907841+0000","last_unstale":"2023-11-17T00:07:03.431743+0000","last_undegraded":"2023-11-17T00:07:03.431743+0000","last_fullsized":"2023-11-17T00:07:03.431743+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:49:13.936360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823548+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:07:02.823548+0000","last_peered":"2023-11-17T00:07:02.823548+0000","last_clean":"2023-11-17T00:07:02.823548+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:07:02.823548+0000","last_undegraded":"2023-11-17T00:07:02.823548+0000","last_fullsized":"2023-11-17T00:07:02.823548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:50:11.202413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825795+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:07:02.825795+0000","last_peered":"2023-11-17T00:07:02.825795+0000","last_clean":"2023-11-17T00:07:02.825795+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:07:02.825795+0000","last_undegraded":"2023-11-17T00:07:02.825795+0000","last_fullsized":"2023-11-17T00:07:02.825795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:13.331240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.832453+0000","last_change":"2023-11-17T00:07:02.832453+0000","last_active":"2023-11-17T00:07:02.832453+0000","last_peered":"2023-11-17T00:07:02.832453+0000","last_clean":"2023-11-17T00:07:02.832453+0000","last_became_active":"2023-11-17T00:07:02.831920+0000","last_became_peered":"2023-11-17T00:07:02.831920+0000","last_unstale":"2023-11-17T00:07:02.832453+0000","last_undegraded":"2023-11-17T00:07:02.832453+0000","last_fullsized":"2023-11-17T00:07:02.832453+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:41.228707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825330+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:07:02.825330+0000","last_peered":"2023-11-17T00:07:02.825330+0000","last_clean":"2023-11-17T00:07:02.825330+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:07:02.825330+0000","last_undegraded":"2023-11-17T00:07:02.825330+0000","last_fullsized":"2023-11-17T00:07:02.825330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:45.221009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825749+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:07:02.825749+0000","last_peered":"2023-11-17T00:07:02.825749+0000","last_clean":"2023-11-17T00:07:02.825749+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:07:02.825749+0000","last_undegraded":"2023-11-17T00:07:02.825749+0000","last_fullsized":"2023-11-17T00:07:02.825749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:18:47.931904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822557+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:07:02.822557+0000","last_peered":"2023-11-17T00:07:02.822557+0000","last_clean":"2023-11-17T00:07:02.822557+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:07:02.822557+0000","last_undegraded":"2023-11-17T00:07:02.822557+0000","last_fullsized":"2023-11-17T00:07:02.822557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:58.808751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826173+0000","last_change":"2023-11-17T00:06:12.638454+0000","last_active":"2023-11-17T00:07:02.826173+0000","last_peered":"2023-11-17T00:07:02.826173+0000","last_clean":"2023-11-17T00:07:02.826173+0000","last_became_active":"2023-11-17T00:06:12.637937+0000","last_became_peered":"2023-11-17T00:06:12.637937+0000","last_unstale":"2023-11-17T00:07:02.826173+0000","last_undegraded":"2023-11-17T00:07:02.826173+0000","last_fullsized":"2023-11-17T00:07:02.826173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:31:28.638284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.828066+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:07:02.828066+0000","last_peered":"2023-11-17T00:07:02.828066+0000","last_clean":"2023-11-17T00:07:02.828066+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:07:02.828066+0000","last_undegraded":"2023-11-17T00:07:02.828066+0000","last_fullsized":"2023-11-17T00:07:02.828066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:04:22.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827817+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:07:02.827817+0000","last_peered":"2023-11-17T00:07:02.827817+0000","last_clean":"2023-11-17T00:07:02.827817+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:07:02.827817+0000","last_undegraded":"2023-11-17T00:07:02.827817+0000","last_fullsized":"2023-11-17T00:07:02.827817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:10.946473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822651+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:07:02.822651+0000","last_peered":"2023-11-17T00:07:02.822651+0000","last_clean":"2023-11-17T00:07:02.822651+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:07:02.822651+0000","last_undegraded":"2023-11-17T00:07:02.822651+0000","last_fullsized":"2023-11-17T00:07:02.822651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:42.813047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825815+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:07:02.825815+0000","last_peered":"2023-11-17T00:07:02.825815+0000","last_clean":"2023-11-17T00:07:02.825815+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:07:02.825815+0000","last_undegraded":"2023-11-17T00:07:02.825815+0000","last_fullsized":"2023-11-17T00:07:02.825815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:42:26.307090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904459+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:07:02.904459+0000","last_peered":"2023-11-17T00:07:02.904459+0000","last_clean":"2023-11-17T00:07:02.904459+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:07:02.904459+0000","last_undegraded":"2023-11-17T00:07:02.904459+0000","last_fullsized":"2023-11-17T00:07:02.904459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:27:34.972172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822134+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:07:02.822134+0000","last_peered":"2023-11-17T00:07:02.822134+0000","last_clean":"2023-11-17T00:07:02.822134+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:07:02.822134+0000","last_undegraded":"2023-11-17T00:07:02.822134+0000","last_fullsized":"2023-11-17T00:07:02.822134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:35:22.231740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826314+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:07:02.826314+0000","last_peered":"2023-11-17T00:07:02.826314+0000","last_clean":"2023-11-17T00:07:02.826314+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:07:02.826314+0000","last_undegraded":"2023-11-17T00:07:02.826314+0000","last_fullsized":"2023-11-17T00:07:02.826314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:13:46.566932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62220,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":121,"num_write":72,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6152,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739000,"statfs":{"total":95995035648,"available":95988736000,"internally_reserved":0,"allocated":983040,"data_stored":494164,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92700000000000005,"5min":1.042,"15min":1.042},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63400000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.38,"5min":0.245,"15min":0.245},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50800000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.438,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97099999999999997,"5min":1.153,"15min":1.153},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.59799999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.95999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60399999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.61199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.476,"15min":1.476},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77300000000000002},"min":{"1min":0.64200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.012,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0289999999999999,"5min":1.429,"15min":1.429},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.65000000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":1.522,"15min":1.522},"last":0.76500000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678049,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":1101824,"data_stored":1141687,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66300000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.496,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82499999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.874,"15min":0.874},"last":0.48699999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.496,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85599999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.747,"15min":0.75},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93600000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":0.996},"last":0.66600000000000004}]},{"osd":9,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":1.115,"15min":1.115},"last":0.749}]},{"osd":11,"last update":"Fri Nov 17 00:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75800000000000001,"15min":0.75600000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76300000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841574,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6284,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738868,"statfs":{"total":95995035648,"available":95988600832,"internally_reserved":0,"allocated":987136,"data_stored":498808,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64400000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83099999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.80600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84799999999999998,"5min":1.093,"15min":1.093},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.77600000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.224,"15min":1.224},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93999999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93400000000000005,"5min":1.28,"15min":1.28},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.85199999999999998,"15min":0.87},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.93100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0649999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.92200000000000004,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0900000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.747}]},{"osd":8,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.98299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":1.004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.01,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95499999999999996,"15min":0.97599999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3660000000000001,"5min":1.51,"15min":1.51},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.83199999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6412,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738740,"statfs":{"total":95995035648,"available":95988469760,"internally_reserved":0,"allocated":987136,"data_stored":494734,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70299999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75600000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.55100000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88500000000000001,"5min":1.139,"15min":1.139},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.44700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85299999999999998,"5min":1.075,"15min":1.075},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.60999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.624,"5min":0.503,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.68700000000000006}]},{"osd":9,"last update":"Fri Nov 17 00:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98399999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.79900000000000004,"15min":0.78500000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.64400000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78900000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.745}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6588,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738564,"statfs":{"total":95995035648,"available":95988289536,"internally_reserved":0,"allocated":1036288,"data_stored":1089176,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64000000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.432,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.57299999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.78600000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.65000000000000002,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":1.083,"15min":1.083},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.54000000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71099999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.49299999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.90900000000000003,"5min":1.046,"15min":1.046},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.69899999999999995},"min":{"1min":0.51100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":1.173,"15min":1.173},"last":0.622}]},{"osd":6,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.73199999999999998,"5min":1.129,"15min":1.129},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.754}]},{"osd":8,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.52600000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":0.84799999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91000000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.73299999999999998,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81399999999999995}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.85799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6400,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738752,"statfs":{"total":95995035648,"available":95988482048,"internally_reserved":0,"allocated":974848,"data_stored":490034,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.623},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.224,"15min":1.224},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68899999999999995,"15min":0.66100000000000003},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.0669999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1679999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.262,"15min":1.262},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.331,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.66900000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79700000000000004,"15min":0.754},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0509999999999999,"5min":1.544,"15min":1.544},"last":0.753},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.76200000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.377,"15min":1.377},"last":0.46999999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.63100000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.472,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Nov 17 00:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.93999999999999995,"15min":0.96299999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.147,"5min":1.429,"15min":1.429},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.99099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1,"15min":1.0169999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.4259999999999999,"5min":1.698,"15min":1.698},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.98299999999999998,"15min":0.99299999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3939999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.879},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.84599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7492,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737660,"statfs":{"total":95995035648,"available":95987363840,"internally_reserved":0,"allocated":1044480,"data_stored":551456,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.30499999999999999,"15min":0.27100000000000002},"max":{"1min":0.89600000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74399999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59299999999999997,"15min":0.54700000000000004},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.63700000000000001,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.378},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.433,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.39600000000000002}]},{"osd":5,"last update":"Fri Nov 17 00:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59199999999999997,"15min":0.55800000000000005},"min":{"1min":0.53600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90900000000000003,"5min":1.395,"15min":1.395},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.123,"15min":1.123},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.54700000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.53900000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.95399999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.71399999999999997}]},{"osd":11,"last update":"Fri Nov 17 00:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.74099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7364,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737788,"statfs":{"total":95995035648,"available":95987494912,"internally_reserved":0,"allocated":978944,"data_stored":490124,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63300000000000001,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.56999999999999995},"min":{"1min":0.54200000000000004,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.627}]},{"osd":2,"last update":"Fri Nov 17 00:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.59499999999999997},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.63600000000000001,"15min":0.58999999999999997},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63900000000000001,"15min":0.58199999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0660000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66500000000000004,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.48199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.64100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.749}]},{"osd":8,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0409999999999999,"5min":1.377,"15min":1.377},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.84799999999999998,"15min":0.81299999999999994},"min":{"1min":0.71899999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1619999999999999,"5min":1.292,"15min":1.292},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.872,"15min":0.87},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0680000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92100000000000004,"15min":0.92500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.147,"5min":1.194,"15min":1.194},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.204,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.93999999999999995},"min":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":0.98599999999999999,"15min":0.96199999999999997},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.343,"5min":4.343,"15min":4.343},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.78400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7492,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737660,"statfs":{"total":95995035648,"available":95987363840,"internally_reserved":0,"allocated":1044480,"data_stored":551456,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53700000000000003,"15min":0.48499999999999999},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.42899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91400000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85499999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.62}]},{"osd":4,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Nov 17 00:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.48999999999999999,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Nov 17 00:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.67300000000000004,"15min":0.628},"min":{"1min":0.60899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67000000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57199999999999995}]},{"osd":8,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.51200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.58799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.753}]},{"osd":9,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.82699999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.59999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.76700000000000002,"15min":0.74299999999999999},"min":{"1min":0.75700000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.77800000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81499999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6724,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738428,"statfs":{"total":95995035648,"available":95988150272,"internally_reserved":0,"allocated":978944,"data_stored":489807,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.41999999999999998},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48999999999999999,"15min":0.44600000000000001},"min":{"1min":0.373,"5min":0.255,"15min":0.255},"max":{"1min":0.62,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.65100000000000002,"15min":0.59799999999999998},"min":{"1min":0.59299999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.60299999999999998},"min":{"1min":0.54400000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69199999999999995,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92800000000000005,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80900000000000005,"5min":1.232,"15min":1.232},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":1.206,"15min":1.206},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.67800000000000005},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.357,"5min":1.359,"15min":1.359},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.755},"min":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2789999999999999,"5min":1.46,"15min":1.46},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.753,"15min":0.71699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.288,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.90000000000000002,"15min":0.92600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.004,"15min":1.034},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2030000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.504}]},{"osd":10,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.98799999999999999,"15min":1.012},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.98999999999999999,"5min":1.496,"15min":1.496},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.012,"15min":1.028},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Fri Nov 17 00:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.79400000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6600,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738552,"statfs":{"total":95995035648,"available":95988277248,"internally_reserved":0,"allocated":983040,"data_stored":494152,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.29899999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.36099999999999999},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54600000000000004,"15min":0.51600000000000001},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Nov 17 00:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59199999999999997,"15min":0.53900000000000003},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45300000000000001}]},{"osd":5,"last update":"Fri Nov 17 00:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.56799999999999995},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84799999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.57499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66700000000000004,"15min":0.64500000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82199999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.52900000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.57399999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96699999999999997,"5min":1.081,"15min":1.081},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.54500000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0169999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73599999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69999999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6720,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738432,"statfs":{"total":95995035648,"available":95988154368,"internally_reserved":0,"allocated":1040384,"data_stored":1084680,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53000000000000003},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86599999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.46200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.41499999999999998,"15min":0.38800000000000001},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.752}]},{"osd":4,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61599999999999999,"15min":0.57899999999999996},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84199999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:07:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83899999999999997,"5min":1.05,"15min":1.05},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.60499999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.046,"15min":1.046},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91000000000000003}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.93300000000000005}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95399999999999996,"5min":1.073,"15min":1.073},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:07:32.718 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-17T00:07:32.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:32 smithi006 ceph-mon[97840]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:32 smithi186 ceph-mon[131487]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:33.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:32 smithi087 ceph-mon[131182]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:33.584 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:34.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:33 smithi186 ceph-mon[131487]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:34.221 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:33 smithi006 ceph-mon[97840]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:34.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:33 smithi087 ceph-mon[131182]: from='client.25006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:34.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:34 smithi006 ceph-mon[97840]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:34 smithi186 ceph-mon[131487]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:34 smithi087 ceph-mon[131182]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 -- 172.21.15.6:0/48124850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f50100ee0 msgr2=0x7f8f50104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 --2- 172.21.15.6:0/48124850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f50100ee0 0x7f8f50104eb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8f400099c0 tx=0x7f8f40009cd0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 -- 172.21.15.6:0/48124850 shutdown_connections 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 --2- 172.21.15.6:0/48124850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f501055e0 0x7f8f5010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 --2- 172.21.15.6:0/48124850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100540 0x7f8f50100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 --2- 172.21.15.6:0/48124850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f50100ee0 0x7f8f50104eb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.393+0000 7f8f5804d700 1 -- 172.21.15.6:0/48124850 >> 172.21.15.6:0/48124850 conn(0x7f8f500fbdc0 msgr2=0x7f8f500fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.394+0000 7f8f5804d700 1 -- 172.21.15.6:0/48124850 shutdown_connections 2023-11-17T00:07:35.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.394+0000 7f8f5804d700 1 -- 172.21.15.6:0/48124850 wait complete. 2023-11-17T00:07:35.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 Processor -- start 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 -- start start 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100ee0 0x7f8f50198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 0x7f8f5019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5019d580 con 0x7f8f501055e0 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5019d6f0 con 0x7f8f50100ee0 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.395+0000 7f8f5804d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5019d8b0 con 0x7f8f50100540 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f55de9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:35.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 0x7f8f5019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f55de9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:40260/0 (socket says 172.21.15.6:40260) 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f55de9700 1 -- 172.21.15.6:0/2124698313 learned_addr learned my addr 172.21.15.6:0/2124698313 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f555e8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100ee0 0x7f8f50198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 -- 172.21.15.6:0/2124698313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 msgr2=0x7f8f501985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 -- 172.21.15.6:0/2124698313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100ee0 msgr2=0x7f8f50198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:35.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100ee0 0x7f8f50198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 -- 172.21.15.6:0/2124698313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400095f0 con 0x7f8f501055e0 2023-11-17T00:07:35.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f55de9700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.396+0000 7f8f565ea700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 0x7f8f5019cf70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800efc0 tx=0x7f8f4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.397+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48018070 con 0x7f8f501055e0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.397+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4800f040 con 0x7f8f501055e0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.397+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800cd40 con 0x7f8f501055e0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.397+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f50110360 con 0x7f8f501055e0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.397+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501101d0 con 0x7f8f501055e0 2023-11-17T00:07:35.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.398+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f501104f0 con 0x7f8f501055e0 2023-11-17T00:07:35.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.400+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8f48003a00 con 0x7f8f501055e0 2023-11-17T00:07:35.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.401+0000 7f8f46ffd700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 0x7f8f3c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:35.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.401+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8f480cc900 con 0x7f8f501055e0 2023-11-17T00:07:35.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.402+0000 7f8f55de9700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 0x7f8f3c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:35.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.402+0000 7f8f55de9700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 0x7f8f3c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c011550 tx=0x7f8f4c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:35.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.405+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8f48095c10 con 0x7f8f501055e0 2023-11-17T00:07:35.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.606+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f500fd480 con 0x7f8f3c0a1260 2023-11-17T00:07:35.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.620+0000 7f8f46ffd700 1 -- 172.21.15.6:0/2124698313 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426573 (secure 0 0 0) 0x7f8f500fd480 con 0x7f8f3c0a1260 2023-11-17T00:07:35.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:35.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 msgr2=0x7f8f3c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:35.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 0x7f8f3c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c011550 tx=0x7f8f4c009260 comp rx=0 tx=0).stop 2023-11-17T00:07:35.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 msgr2=0x7f8f5019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:35.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 0x7f8f5019cf70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800efc0 tx=0x7f8f4800c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 shutdown_connections 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8f3c0a1260 0x7f8f3c0a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f501055e0 0x7f8f5019cf70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f50100540 0x7f8f501985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 --2- 172.21.15.6:0/2124698313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f50100ee0 0x7f8f50198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:35.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 >> 172.21.15.6:0/2124698313 conn(0x7f8f500fbdc0 msgr2=0x7f8f5010b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:35.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 shutdown_connections 2023-11-17T00:07:35.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:35.625+0000 7f8f5804d700 1 -- 172.21.15.6:0/2124698313 wait complete. 2023-11-17T00:07:35.641 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:07:36.392 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2023-11-17T00:07:33.760570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652834,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":245,"num_write":255,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80752,"kb_used_data":11856,"kb_used_omap":0,"kb_used_meta":68800,"kb_avail":1124861072,"statfs":{"total":1151940427776,"available":1151857737728,"internally_reserved":0,"allocated":12140544,"data_stored":7870278,"data_compressed":318714,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70451200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003353"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826846+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:07:02.826846+0000","last_peered":"2023-11-17T00:07:02.826846+0000","last_clean":"2023-11-17T00:07:02.826846+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:07:02.826846+0000","last_undegraded":"2023-11-17T00:07:02.826846+0000","last_fullsized":"2023-11-17T00:07:02.826846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:29.635832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826015+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:07:02.826015+0000","last_peered":"2023-11-17T00:07:02.826015+0000","last_clean":"2023-11-17T00:07:02.826015+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:07:02.826015+0000","last_undegraded":"2023-11-17T00:07:02.826015+0000","last_fullsized":"2023-11-17T00:07:02.826015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:40.711215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825749+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:07:02.825749+0000","last_peered":"2023-11-17T00:07:02.825749+0000","last_clean":"2023-11-17T00:07:02.825749+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:07:02.825749+0000","last_undegraded":"2023-11-17T00:07:02.825749+0000","last_fullsized":"2023-11-17T00:07:02.825749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:28.698003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905999+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:07:02.905999+0000","last_peered":"2023-11-17T00:07:02.905999+0000","last_clean":"2023-11-17T00:07:02.905999+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:07:02.905999+0000","last_undegraded":"2023-11-17T00:07:02.905999+0000","last_fullsized":"2023-11-17T00:07:02.905999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:48:34.307597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826002+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:07:02.826002+0000","last_peered":"2023-11-17T00:07:02.826002+0000","last_clean":"2023-11-17T00:07:02.826002+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:07:02.826002+0000","last_undegraded":"2023-11-17T00:07:02.826002+0000","last_fullsized":"2023-11-17T00:07:02.826002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:52:25.271013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834121+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:07:02.834121+0000","last_peered":"2023-11-17T00:07:02.834121+0000","last_clean":"2023-11-17T00:07:02.834121+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:07:02.834121+0000","last_undegraded":"2023-11-17T00:07:02.834121+0000","last_fullsized":"2023-11-17T00:07:02.834121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:16:58.736045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822474+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:07:02.822474+0000","last_peered":"2023-11-17T00:07:02.822474+0000","last_clean":"2023-11-17T00:07:02.822474+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:07:02.822474+0000","last_undegraded":"2023-11-17T00:07:02.822474+0000","last_fullsized":"2023-11-17T00:07:02.822474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:52:21.436852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827042+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:07:02.827042+0000","last_peered":"2023-11-17T00:07:02.827042+0000","last_clean":"2023-11-17T00:07:02.827042+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:07:02.827042+0000","last_undegraded":"2023-11-17T00:07:02.827042+0000","last_fullsized":"2023-11-17T00:07:02.827042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:31:42.654675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905836+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:07:02.905836+0000","last_peered":"2023-11-17T00:07:02.905836+0000","last_clean":"2023-11-17T00:07:02.905836+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:07:02.905836+0000","last_undegraded":"2023-11-17T00:07:02.905836+0000","last_fullsized":"2023-11-17T00:07:02.905836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:31:48.399844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834537+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:07:02.834537+0000","last_peered":"2023-11-17T00:07:02.834537+0000","last_clean":"2023-11-17T00:07:02.834537+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:07:02.834537+0000","last_undegraded":"2023-11-17T00:07:02.834537+0000","last_fullsized":"2023-11-17T00:07:02.834537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:41.253445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823197+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:07:02.823197+0000","last_peered":"2023-11-17T00:07:02.823197+0000","last_clean":"2023-11-17T00:07:02.823197+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:07:02.823197+0000","last_undegraded":"2023-11-17T00:07:02.823197+0000","last_fullsized":"2023-11-17T00:07:02.823197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:18:43.612147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905815+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:07:02.905815+0000","last_peered":"2023-11-17T00:07:02.905815+0000","last_clean":"2023-11-17T00:07:02.905815+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:07:02.905815+0000","last_undegraded":"2023-11-17T00:07:02.905815+0000","last_fullsized":"2023-11-17T00:07:02.905815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:52:55.579056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822297+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:07:02.822297+0000","last_peered":"2023-11-17T00:07:02.822297+0000","last_clean":"2023-11-17T00:07:02.822297+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:07:02.822297+0000","last_undegraded":"2023-11-17T00:07:02.822297+0000","last_fullsized":"2023-11-17T00:07:02.822297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:04:17.073703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825103+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:07:02.825103+0000","last_peered":"2023-11-17T00:07:02.825103+0000","last_clean":"2023-11-17T00:07:02.825103+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:07:02.825103+0000","last_undegraded":"2023-11-17T00:07:02.825103+0000","last_fullsized":"2023-11-17T00:07:02.825103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:28:42.750181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825495+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:07:02.825495+0000","last_peered":"2023-11-17T00:07:02.825495+0000","last_clean":"2023-11-17T00:07:02.825495+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:07:02.825495+0000","last_undegraded":"2023-11-17T00:07:02.825495+0000","last_fullsized":"2023-11-17T00:07:02.825495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:36:00.587367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904968+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:07:02.904968+0000","last_peered":"2023-11-17T00:07:02.904968+0000","last_clean":"2023-11-17T00:07:02.904968+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:07:02.904968+0000","last_undegraded":"2023-11-17T00:07:02.904968+0000","last_fullsized":"2023-11-17T00:07:02.904968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:15.663690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826706+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:07:02.826706+0000","last_peered":"2023-11-17T00:07:02.826706+0000","last_clean":"2023-11-17T00:07:02.826706+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:07:02.826706+0000","last_undegraded":"2023-11-17T00:07:02.826706+0000","last_fullsized":"2023-11-17T00:07:02.826706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:07:48.794377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904844+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:07:02.904844+0000","last_peered":"2023-11-17T00:07:02.904844+0000","last_clean":"2023-11-17T00:07:02.904844+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:07:02.904844+0000","last_undegraded":"2023-11-17T00:07:02.904844+0000","last_fullsized":"2023-11-17T00:07:02.904844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:48.005898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827922+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:07:02.827922+0000","last_peered":"2023-11-17T00:07:02.827922+0000","last_clean":"2023-11-17T00:07:02.827922+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:07:02.827922+0000","last_undegraded":"2023-11-17T00:07:02.827922+0000","last_fullsized":"2023-11-17T00:07:02.827922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:06:26.414630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826839+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:07:02.826839+0000","last_peered":"2023-11-17T00:07:02.826839+0000","last_clean":"2023-11-17T00:07:02.826839+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:07:02.826839+0000","last_undegraded":"2023-11-17T00:07:02.826839+0000","last_fullsized":"2023-11-17T00:07:02.826839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:36:59.477340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904660+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:07:02.904660+0000","last_peered":"2023-11-17T00:07:02.904660+0000","last_clean":"2023-11-17T00:07:02.904660+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:07:02.904660+0000","last_undegraded":"2023-11-17T00:07:02.904660+0000","last_fullsized":"2023-11-17T00:07:02.904660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:25.085518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827626+0000","last_change":"2023-11-17T00:06:12.628519+0000","last_active":"2023-11-17T00:07:02.827626+0000","last_peered":"2023-11-17T00:07:02.827626+0000","last_clean":"2023-11-17T00:07:02.827626+0000","last_became_active":"2023-11-17T00:06:12.627360+0000","last_became_peered":"2023-11-17T00:06:12.627360+0000","last_unstale":"2023-11-17T00:07:02.827626+0000","last_undegraded":"2023-11-17T00:07:02.827626+0000","last_fullsized":"2023-11-17T00:07:02.827626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:09:51.885082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904221+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:07:02.904221+0000","last_peered":"2023-11-17T00:07:02.904221+0000","last_clean":"2023-11-17T00:07:02.904221+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:07:02.904221+0000","last_undegraded":"2023-11-17T00:07:02.904221+0000","last_fullsized":"2023-11-17T00:07:02.904221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:51:29.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827768+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:07:02.827768+0000","last_peered":"2023-11-17T00:07:02.827768+0000","last_clean":"2023-11-17T00:07:02.827768+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:07:02.827768+0000","last_undegraded":"2023-11-17T00:07:02.827768+0000","last_fullsized":"2023-11-17T00:07:02.827768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:23.232685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826329+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:07:02.826329+0000","last_peered":"2023-11-17T00:07:02.826329+0000","last_clean":"2023-11-17T00:07:02.826329+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:07:02.826329+0000","last_undegraded":"2023-11-17T00:07:02.826329+0000","last_fullsized":"2023-11-17T00:07:02.826329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:49:43.218061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905006+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:07:02.905006+0000","last_peered":"2023-11-17T00:07:02.905006+0000","last_clean":"2023-11-17T00:07:02.905006+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:07:02.905006+0000","last_undegraded":"2023-11-17T00:07:02.905006+0000","last_fullsized":"2023-11-17T00:07:02.905006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:14:06.574179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826705+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:07:02.826705+0000","last_peered":"2023-11-17T00:07:02.826705+0000","last_clean":"2023-11-17T00:07:02.826705+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:07:02.826705+0000","last_undegraded":"2023-11-17T00:07:02.826705+0000","last_fullsized":"2023-11-17T00:07:02.826705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:02:22.741567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904169+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:07:02.904169+0000","last_peered":"2023-11-17T00:07:02.904169+0000","last_clean":"2023-11-17T00:07:02.904169+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:07:02.904169+0000","last_undegraded":"2023-11-17T00:07:02.904169+0000","last_fullsized":"2023-11-17T00:07:02.904169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:31:07.179458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904738+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:07:02.904738+0000","last_peered":"2023-11-17T00:07:02.904738+0000","last_clean":"2023-11-17T00:07:02.904738+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:07:02.904738+0000","last_undegraded":"2023-11-17T00:07:02.904738+0000","last_fullsized":"2023-11-17T00:07:02.904738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:39:24.141254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825793+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:07:02.825793+0000","last_peered":"2023-11-17T00:07:02.825793+0000","last_clean":"2023-11-17T00:07:02.825793+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:07:02.825793+0000","last_undegraded":"2023-11-17T00:07:02.825793+0000","last_fullsized":"2023-11-17T00:07:02.825793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:29:35.519337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823032+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:07:02.823032+0000","last_peered":"2023-11-17T00:07:02.823032+0000","last_clean":"2023-11-17T00:07:02.823032+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:07:02.823032+0000","last_undegraded":"2023-11-17T00:07:02.823032+0000","last_fullsized":"2023-11-17T00:07:02.823032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:04.449613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827448+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:07:02.827448+0000","last_peered":"2023-11-17T00:07:02.827448+0000","last_clean":"2023-11-17T00:07:02.827448+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:07:02.827448+0000","last_undegraded":"2023-11-17T00:07:02.827448+0000","last_fullsized":"2023-11-17T00:07:02.827448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:42:04.048485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823255+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:07:02.823255+0000","last_peered":"2023-11-17T00:07:02.823255+0000","last_clean":"2023-11-17T00:07:02.823255+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:07:02.823255+0000","last_undegraded":"2023-11-17T00:07:02.823255+0000","last_fullsized":"2023-11-17T00:07:02.823255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:55:03.575265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827611+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:07:02.827611+0000","last_peered":"2023-11-17T00:07:02.827611+0000","last_clean":"2023-11-17T00:07:02.827611+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:07:02.827611+0000","last_undegraded":"2023-11-17T00:07:02.827611+0000","last_fullsized":"2023-11-17T00:07:02.827611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:25:30.932439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827552+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:07:02.827552+0000","last_peered":"2023-11-17T00:07:02.827552+0000","last_clean":"2023-11-17T00:07:02.827552+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:07:02.827552+0000","last_undegraded":"2023-11-17T00:07:02.827552+0000","last_fullsized":"2023-11-17T00:07:02.827552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:54:56.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834106+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:07:02.834106+0000","last_peered":"2023-11-17T00:07:02.834106+0000","last_clean":"2023-11-17T00:07:02.834106+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:07:02.834106+0000","last_undegraded":"2023-11-17T00:07:02.834106+0000","last_fullsized":"2023-11-17T00:07:02.834106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:36:35.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.824231+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:07:02.824231+0000","last_peered":"2023-11-17T00:07:02.824231+0000","last_clean":"2023-11-17T00:07:02.824231+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:07:02.824231+0000","last_undegraded":"2023-11-17T00:07:02.824231+0000","last_fullsized":"2023-11-17T00:07:02.824231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:31:30.495283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.907032+0000","last_change":"2023-11-17T00:07:02.907032+0000","last_active":"2023-11-17T00:07:02.907032+0000","last_peered":"2023-11-17T00:07:02.907032+0000","last_clean":"2023-11-17T00:07:02.907032+0000","last_became_active":"2023-11-17T00:07:02.906576+0000","last_became_peered":"2023-11-17T00:07:02.906576+0000","last_unstale":"2023-11-17T00:07:02.907032+0000","last_undegraded":"2023-11-17T00:07:02.907032+0000","last_fullsized":"2023-11-17T00:07:02.907032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:17:37.002477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821900+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:07:02.821900+0000","last_peered":"2023-11-17T00:07:02.821900+0000","last_clean":"2023-11-17T00:07:02.821900+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:07:02.821900+0000","last_undegraded":"2023-11-17T00:07:02.821900+0000","last_fullsized":"2023-11-17T00:07:02.821900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:25:54.909513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826999+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:07:02.826999+0000","last_peered":"2023-11-17T00:07:02.826999+0000","last_clean":"2023-11-17T00:07:02.826999+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:07:02.826999+0000","last_undegraded":"2023-11-17T00:07:02.826999+0000","last_fullsized":"2023-11-17T00:07:02.826999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:23:16.172360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826053+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:07:02.826053+0000","last_peered":"2023-11-17T00:07:02.826053+0000","last_clean":"2023-11-17T00:07:02.826053+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:07:02.826053+0000","last_undegraded":"2023-11-17T00:07:02.826053+0000","last_fullsized":"2023-11-17T00:07:02.826053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:41.604524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822579+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:07:02.822579+0000","last_peered":"2023-11-17T00:07:02.822579+0000","last_clean":"2023-11-17T00:07:02.822579+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:07:02.822579+0000","last_undegraded":"2023-11-17T00:07:02.822579+0000","last_fullsized":"2023-11-17T00:07:02.822579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:33:24.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903950+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:07:02.903950+0000","last_peered":"2023-11-17T00:07:02.903950+0000","last_clean":"2023-11-17T00:07:02.903950+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:07:02.903950+0000","last_undegraded":"2023-11-17T00:07:02.903950+0000","last_fullsized":"2023-11-17T00:07:02.903950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:26.168940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826025+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:07:02.826025+0000","last_peered":"2023-11-17T00:07:02.826025+0000","last_clean":"2023-11-17T00:07:02.826025+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:07:02.826025+0000","last_undegraded":"2023-11-17T00:07:02.826025+0000","last_fullsized":"2023-11-17T00:07:02.826025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:22:22.019983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834455+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:07:02.834455+0000","last_peered":"2023-11-17T00:07:02.834455+0000","last_clean":"2023-11-17T00:07:02.834455+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:07:02.834455+0000","last_undegraded":"2023-11-17T00:07:02.834455+0000","last_fullsized":"2023-11-17T00:07:02.834455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:44:14.351304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905159+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:07:02.905159+0000","last_peered":"2023-11-17T00:07:02.905159+0000","last_clean":"2023-11-17T00:07:02.905159+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:07:02.905159+0000","last_undegraded":"2023-11-17T00:07:02.905159+0000","last_fullsized":"2023-11-17T00:07:02.905159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:05:24.835195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903836+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:07:02.903836+0000","last_peered":"2023-11-17T00:07:02.903836+0000","last_clean":"2023-11-17T00:07:02.903836+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:07:02.903836+0000","last_undegraded":"2023-11-17T00:07:02.903836+0000","last_fullsized":"2023-11-17T00:07:02.903836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:53:26.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823087+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:07:02.823087+0000","last_peered":"2023-11-17T00:07:02.823087+0000","last_clean":"2023-11-17T00:07:02.823087+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:07:02.823087+0000","last_undegraded":"2023-11-17T00:07:02.823087+0000","last_fullsized":"2023-11-17T00:07:02.823087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:33:35.310318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826437+0000","last_change":"2023-11-17T00:06:12.638172+0000","last_active":"2023-11-17T00:07:02.826437+0000","last_peered":"2023-11-17T00:07:02.826437+0000","last_clean":"2023-11-17T00:07:02.826437+0000","last_became_active":"2023-11-17T00:06:12.637227+0000","last_became_peered":"2023-11-17T00:06:12.637227+0000","last_unstale":"2023-11-17T00:07:02.826437+0000","last_undegraded":"2023-11-17T00:07:02.826437+0000","last_fullsized":"2023-11-17T00:07:02.826437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:48:57.804467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905310+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:07:02.905310+0000","last_peered":"2023-11-17T00:07:02.905310+0000","last_clean":"2023-11-17T00:07:02.905310+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:07:02.905310+0000","last_undegraded":"2023-11-17T00:07:02.905310+0000","last_fullsized":"2023-11-17T00:07:02.905310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:13.073808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822885+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:07:02.822885+0000","last_peered":"2023-11-17T00:07:02.822885+0000","last_clean":"2023-11-17T00:07:02.822885+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:07:02.822885+0000","last_undegraded":"2023-11-17T00:07:02.822885+0000","last_fullsized":"2023-11-17T00:07:02.822885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:56:49.301970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904320+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:07:02.904320+0000","last_peered":"2023-11-17T00:07:02.904320+0000","last_clean":"2023-11-17T00:07:02.904320+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:07:02.904320+0000","last_undegraded":"2023-11-17T00:07:02.904320+0000","last_fullsized":"2023-11-17T00:07:02.904320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:26.693598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.903985+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:07:02.903985+0000","last_peered":"2023-11-17T00:07:02.903985+0000","last_clean":"2023-11-17T00:07:02.903985+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:07:02.903985+0000","last_undegraded":"2023-11-17T00:07:02.903985+0000","last_fullsized":"2023-11-17T00:07:02.903985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:25.639542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:32.932322+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:07:32.932322+0000","last_peered":"2023-11-17T00:07:32.932322+0000","last_clean":"2023-11-17T00:07:32.932322+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:07:32.932322+0000","last_undegraded":"2023-11-17T00:07:32.932322+0000","last_fullsized":"2023-11-17T00:07:32.932322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:16.841523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61422,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":121,"num_read_kb":120,"num_write":29,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826378+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:07:02.826378+0000","last_peered":"2023-11-17T00:07:02.826378+0000","last_clean":"2023-11-17T00:07:02.826378+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:07:02.826378+0000","last_undegraded":"2023-11-17T00:07:02.826378+0000","last_fullsized":"2023-11-17T00:07:02.826378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:28:55.207231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823519+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:07:02.823519+0000","last_peered":"2023-11-17T00:07:02.823519+0000","last_clean":"2023-11-17T00:07:02.823519+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:07:02.823519+0000","last_undegraded":"2023-11-17T00:07:02.823519+0000","last_fullsized":"2023-11-17T00:07:02.823519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:36:08.220274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:03.429143+0000","last_change":"2023-11-17T00:07:03.428692+0000","last_active":"2023-11-17T00:07:03.429143+0000","last_peered":"2023-11-17T00:07:03.429143+0000","last_clean":"2023-11-17T00:07:03.429143+0000","last_became_active":"2023-11-17T00:07:02.832461+0000","last_became_peered":"2023-11-17T00:07:02.832461+0000","last_unstale":"2023-11-17T00:07:03.429143+0000","last_undegraded":"2023-11-17T00:07:03.429143+0000","last_fullsized":"2023-11-17T00:07:03.429143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:24.423317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822030+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:07:02.822030+0000","last_peered":"2023-11-17T00:07:02.822030+0000","last_clean":"2023-11-17T00:07:02.822030+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:07:02.822030+0000","last_undegraded":"2023-11-17T00:07:02.822030+0000","last_fullsized":"2023-11-17T00:07:02.822030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:11:44.312321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.831797+0000","last_change":"2023-11-17T00:07:02.831797+0000","last_active":"2023-11-17T00:07:02.831797+0000","last_peered":"2023-11-17T00:07:02.831797+0000","last_clean":"2023-11-17T00:07:02.831797+0000","last_became_active":"2023-11-17T00:07:02.831339+0000","last_became_peered":"2023-11-17T00:07:02.831339+0000","last_unstale":"2023-11-17T00:07:02.831797+0000","last_undegraded":"2023-11-17T00:07:02.831797+0000","last_fullsized":"2023-11-17T00:07:02.831797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:58:34.477233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823406+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:07:02.823406+0000","last_peered":"2023-11-17T00:07:02.823406+0000","last_clean":"2023-11-17T00:07:02.823406+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:07:02.823406+0000","last_undegraded":"2023-11-17T00:07:02.823406+0000","last_fullsized":"2023-11-17T00:07:02.823406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:39:41.389091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905543+0000","last_change":"2023-11-17T00:07:02.905543+0000","last_active":"2023-11-17T00:07:02.905543+0000","last_peered":"2023-11-17T00:07:02.905543+0000","last_clean":"2023-11-17T00:07:02.905543+0000","last_became_active":"2023-11-17T00:07:02.905087+0000","last_became_peered":"2023-11-17T00:07:02.905087+0000","last_unstale":"2023-11-17T00:07:02.905543+0000","last_undegraded":"2023-11-17T00:07:02.905543+0000","last_fullsized":"2023-11-17T00:07:02.905543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:40:53.411044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904498+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:07:02.904498+0000","last_peered":"2023-11-17T00:07:02.904498+0000","last_clean":"2023-11-17T00:07:02.904498+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:07:02.904498+0000","last_undegraded":"2023-11-17T00:07:02.904498+0000","last_fullsized":"2023-11-17T00:07:02.904498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:10:09.796313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905570+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:07:02.905570+0000","last_peered":"2023-11-17T00:07:02.905570+0000","last_clean":"2023-11-17T00:07:02.905570+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:07:02.905570+0000","last_undegraded":"2023-11-17T00:07:02.905570+0000","last_fullsized":"2023-11-17T00:07:02.905570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:08.691707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823381+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:07:02.823381+0000","last_peered":"2023-11-17T00:07:02.823381+0000","last_clean":"2023-11-17T00:07:02.823381+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:07:02.823381+0000","last_undegraded":"2023-11-17T00:07:02.823381+0000","last_fullsized":"2023-11-17T00:07:02.823381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:29:14.359493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822441+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:07:02.822441+0000","last_peered":"2023-11-17T00:07:02.822441+0000","last_clean":"2023-11-17T00:07:02.822441+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:07:02.822441+0000","last_undegraded":"2023-11-17T00:07:02.822441+0000","last_fullsized":"2023-11-17T00:07:02.822441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:02.079300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825837+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:07:02.825837+0000","last_peered":"2023-11-17T00:07:02.825837+0000","last_clean":"2023-11-17T00:07:02.825837+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:07:02.825837+0000","last_undegraded":"2023-11-17T00:07:02.825837+0000","last_fullsized":"2023-11-17T00:07:02.825837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:46:21.300209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822789+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:07:02.822789+0000","last_peered":"2023-11-17T00:07:02.822789+0000","last_clean":"2023-11-17T00:07:02.822789+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:07:02.822789+0000","last_undegraded":"2023-11-17T00:07:02.822789+0000","last_fullsized":"2023-11-17T00:07:02.822789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:46:38.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822639+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:07:02.822639+0000","last_peered":"2023-11-17T00:07:02.822639+0000","last_clean":"2023-11-17T00:07:02.822639+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:07:02.822639+0000","last_undegraded":"2023-11-17T00:07:02.822639+0000","last_fullsized":"2023-11-17T00:07:02.822639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:20.570854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826407+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:07:02.826407+0000","last_peered":"2023-11-17T00:07:02.826407+0000","last_clean":"2023-11-17T00:07:02.826407+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:07:02.826407+0000","last_undegraded":"2023-11-17T00:07:02.826407+0000","last_fullsized":"2023-11-17T00:07:02.826407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:19.334144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827339+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:07:02.827339+0000","last_peered":"2023-11-17T00:07:02.827339+0000","last_clean":"2023-11-17T00:07:02.827339+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:07:02.827339+0000","last_undegraded":"2023-11-17T00:07:02.827339+0000","last_fullsized":"2023-11-17T00:07:02.827339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:59.198507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827452+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:07:02.827452+0000","last_peered":"2023-11-17T00:07:02.827452+0000","last_clean":"2023-11-17T00:07:02.827452+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:07:02.827452+0000","last_undegraded":"2023-11-17T00:07:02.827452+0000","last_fullsized":"2023-11-17T00:07:02.827452+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:02:41.038095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904302+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:07:02.904302+0000","last_peered":"2023-11-17T00:07:02.904302+0000","last_clean":"2023-11-17T00:07:02.904302+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:07:02.904302+0000","last_undegraded":"2023-11-17T00:07:02.904302+0000","last_fullsized":"2023-11-17T00:07:02.904302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:33:31.825193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834456+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:07:02.834456+0000","last_peered":"2023-11-17T00:07:02.834456+0000","last_clean":"2023-11-17T00:07:02.834456+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:07:02.834456+0000","last_undegraded":"2023-11-17T00:07:02.834456+0000","last_fullsized":"2023-11-17T00:07:02.834456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:07.757713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822319+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:07:02.822319+0000","last_peered":"2023-11-17T00:07:02.822319+0000","last_clean":"2023-11-17T00:07:02.822319+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:07:02.822319+0000","last_undegraded":"2023-11-17T00:07:02.822319+0000","last_fullsized":"2023-11-17T00:07:02.822319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:31.546886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822628+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:07:02.822628+0000","last_peered":"2023-11-17T00:07:02.822628+0000","last_clean":"2023-11-17T00:07:02.822628+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:07:02.822628+0000","last_undegraded":"2023-11-17T00:07:02.822628+0000","last_fullsized":"2023-11-17T00:07:02.822628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:57.597621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.834235+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:07:02.834235+0000","last_peered":"2023-11-17T00:07:02.834235+0000","last_clean":"2023-11-17T00:07:02.834235+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:07:02.834235+0000","last_undegraded":"2023-11-17T00:07:02.834235+0000","last_fullsized":"2023-11-17T00:07:02.834235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:35:18.172438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822421+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:07:02.822421+0000","last_peered":"2023-11-17T00:07:02.822421+0000","last_clean":"2023-11-17T00:07:02.822421+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:07:02.822421+0000","last_undegraded":"2023-11-17T00:07:02.822421+0000","last_fullsized":"2023-11-17T00:07:02.822421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:25:55.041206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822811+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:07:02.822811+0000","last_peered":"2023-11-17T00:07:02.822811+0000","last_clean":"2023-11-17T00:07:02.822811+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:07:02.822811+0000","last_undegraded":"2023-11-17T00:07:02.822811+0000","last_fullsized":"2023-11-17T00:07:02.822811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:11:58.695080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821790+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:07:02.821790+0000","last_peered":"2023-11-17T00:07:02.821790+0000","last_clean":"2023-11-17T00:07:02.821790+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:07:02.821790+0000","last_undegraded":"2023-11-17T00:07:02.821790+0000","last_fullsized":"2023-11-17T00:07:02.821790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:19:50.407968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822812+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:07:02.822812+0000","last_peered":"2023-11-17T00:07:02.822812+0000","last_clean":"2023-11-17T00:07:02.822812+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:07:02.822812+0000","last_undegraded":"2023-11-17T00:07:02.822812+0000","last_fullsized":"2023-11-17T00:07:02.822812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:40:47.701107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826430+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:07:02.826430+0000","last_peered":"2023-11-17T00:07:02.826430+0000","last_clean":"2023-11-17T00:07:02.826430+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:07:02.826430+0000","last_undegraded":"2023-11-17T00:07:02.826430+0000","last_fullsized":"2023-11-17T00:07:02.826430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:31:19.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905156+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:07:02.905156+0000","last_peered":"2023-11-17T00:07:02.905156+0000","last_clean":"2023-11-17T00:07:02.905156+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:07:02.905156+0000","last_undegraded":"2023-11-17T00:07:02.905156+0000","last_fullsized":"2023-11-17T00:07:02.905156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:59:31.001093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822513+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:07:02.822513+0000","last_peered":"2023-11-17T00:07:02.822513+0000","last_clean":"2023-11-17T00:07:02.822513+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:07:02.822513+0000","last_undegraded":"2023-11-17T00:07:02.822513+0000","last_fullsized":"2023-11-17T00:07:02.822513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:02:56.262488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823767+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:07:02.823767+0000","last_peered":"2023-11-17T00:07:02.823767+0000","last_clean":"2023-11-17T00:07:02.823767+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:07:02.823767+0000","last_undegraded":"2023-11-17T00:07:02.823767+0000","last_fullsized":"2023-11-17T00:07:02.823767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:02:15.925740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.821770+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:07:02.821770+0000","last_peered":"2023-11-17T00:07:02.821770+0000","last_clean":"2023-11-17T00:07:02.821770+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:07:02.821770+0000","last_undegraded":"2023-11-17T00:07:02.821770+0000","last_fullsized":"2023-11-17T00:07:02.821770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:57:19.724909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906620+0000","last_change":"2023-11-17T00:07:02.906620+0000","last_active":"2023-11-17T00:07:02.906620+0000","last_peered":"2023-11-17T00:07:02.906620+0000","last_clean":"2023-11-17T00:07:02.906620+0000","last_became_active":"2023-11-17T00:07:02.906237+0000","last_became_peered":"2023-11-17T00:07:02.906237+0000","last_unstale":"2023-11-17T00:07:02.906620+0000","last_undegraded":"2023-11-17T00:07:02.906620+0000","last_fullsized":"2023-11-17T00:07:02.906620+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:45:43.859463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827046+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:07:02.827046+0000","last_peered":"2023-11-17T00:07:02.827046+0000","last_clean":"2023-11-17T00:07:02.827046+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:07:02.827046+0000","last_undegraded":"2023-11-17T00:07:02.827046+0000","last_fullsized":"2023-11-17T00:07:02.827046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:45:13.061257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826500+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:07:02.826500+0000","last_peered":"2023-11-17T00:07:02.826500+0000","last_clean":"2023-11-17T00:07:02.826500+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:07:02.826500+0000","last_undegraded":"2023-11-17T00:07:02.826500+0000","last_fullsized":"2023-11-17T00:07:02.826500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:47:36.362600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822518+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:07:02.822518+0000","last_peered":"2023-11-17T00:07:02.822518+0000","last_clean":"2023-11-17T00:07:02.822518+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:07:02.822518+0000","last_undegraded":"2023-11-17T00:07:02.822518+0000","last_fullsized":"2023-11-17T00:07:02.822518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:28:46.236875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826859+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:07:02.826859+0000","last_peered":"2023-11-17T00:07:02.826859+0000","last_clean":"2023-11-17T00:07:02.826859+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:07:02.826859+0000","last_undegraded":"2023-11-17T00:07:02.826859+0000","last_fullsized":"2023-11-17T00:07:02.826859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:54:22.051875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904721+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:07:02.904721+0000","last_peered":"2023-11-17T00:07:02.904721+0000","last_clean":"2023-11-17T00:07:02.904721+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:07:02.904721+0000","last_undegraded":"2023-11-17T00:07:02.904721+0000","last_fullsized":"2023-11-17T00:07:02.904721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:47:50.659334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827650+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:07:02.827650+0000","last_peered":"2023-11-17T00:07:02.827650+0000","last_clean":"2023-11-17T00:07:02.827650+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:07:02.827650+0000","last_undegraded":"2023-11-17T00:07:02.827650+0000","last_fullsized":"2023-11-17T00:07:02.827650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:56:54.546763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826712+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:07:02.826712+0000","last_peered":"2023-11-17T00:07:02.826712+0000","last_clean":"2023-11-17T00:07:02.826712+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:07:02.826712+0000","last_undegraded":"2023-11-17T00:07:02.826712+0000","last_fullsized":"2023-11-17T00:07:02.826712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:53.773020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906453+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:07:02.906453+0000","last_peered":"2023-11-17T00:07:02.906453+0000","last_clean":"2023-11-17T00:07:02.906453+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:07:02.906453+0000","last_undegraded":"2023-11-17T00:07:02.906453+0000","last_fullsized":"2023-11-17T00:07:02.906453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:24:35.297870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826152+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:07:02.826152+0000","last_peered":"2023-11-17T00:07:02.826152+0000","last_clean":"2023-11-17T00:07:02.826152+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:07:02.826152+0000","last_undegraded":"2023-11-17T00:07:02.826152+0000","last_fullsized":"2023-11-17T00:07:02.826152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:58.836590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906353+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:07:02.906353+0000","last_peered":"2023-11-17T00:07:02.906353+0000","last_clean":"2023-11-17T00:07:02.906353+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:07:02.906353+0000","last_undegraded":"2023-11-17T00:07:02.906353+0000","last_fullsized":"2023-11-17T00:07:02.906353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:24:07.960782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905119+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:07:02.905119+0000","last_peered":"2023-11-17T00:07:02.905119+0000","last_clean":"2023-11-17T00:07:02.905119+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:07:02.905119+0000","last_undegraded":"2023-11-17T00:07:02.905119+0000","last_fullsized":"2023-11-17T00:07:02.905119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:18:12.956129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825020+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:07:02.825020+0000","last_peered":"2023-11-17T00:07:02.825020+0000","last_clean":"2023-11-17T00:07:02.825020+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:07:02.825020+0000","last_undegraded":"2023-11-17T00:07:02.825020+0000","last_fullsized":"2023-11-17T00:07:02.825020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:41:11.727361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904629+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:07:02.904629+0000","last_peered":"2023-11-17T00:07:02.904629+0000","last_clean":"2023-11-17T00:07:02.904629+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:07:02.904629+0000","last_undegraded":"2023-11-17T00:07:02.904629+0000","last_fullsized":"2023-11-17T00:07:02.904629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:48:14.535390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825694+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:07:02.825694+0000","last_peered":"2023-11-17T00:07:02.825694+0000","last_clean":"2023-11-17T00:07:02.825694+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:07:02.825694+0000","last_undegraded":"2023-11-17T00:07:02.825694+0000","last_fullsized":"2023-11-17T00:07:02.825694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:43:12.873004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825643+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:07:02.825643+0000","last_peered":"2023-11-17T00:07:02.825643+0000","last_clean":"2023-11-17T00:07:02.825643+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:07:02.825643+0000","last_undegraded":"2023-11-17T00:07:02.825643+0000","last_fullsized":"2023-11-17T00:07:02.825643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:13:29.632520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905501+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:07:02.905501+0000","last_peered":"2023-11-17T00:07:02.905501+0000","last_clean":"2023-11-17T00:07:02.905501+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:07:02.905501+0000","last_undegraded":"2023-11-17T00:07:02.905501+0000","last_fullsized":"2023-11-17T00:07:02.905501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:31:46.721192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.832991+0000","last_change":"2023-11-17T00:07:02.831541+0000","last_active":"2023-11-17T00:07:02.832991+0000","last_peered":"2023-11-17T00:07:02.832991+0000","last_clean":"2023-11-17T00:07:02.832991+0000","last_became_active":"2023-11-17T00:07:02.831030+0000","last_became_peered":"2023-11-17T00:07:02.831030+0000","last_unstale":"2023-11-17T00:07:02.832991+0000","last_undegraded":"2023-11-17T00:07:02.832991+0000","last_fullsized":"2023-11-17T00:07:02.832991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:33:02.117643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827288+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:07:02.827288+0000","last_peered":"2023-11-17T00:07:02.827288+0000","last_clean":"2023-11-17T00:07:02.827288+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:07:02.827288+0000","last_undegraded":"2023-11-17T00:07:02.827288+0000","last_fullsized":"2023-11-17T00:07:02.827288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:07:19.757734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.905512+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:07:02.905512+0000","last_peered":"2023-11-17T00:07:02.905512+0000","last_clean":"2023-11-17T00:07:02.905512+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:07:02.905512+0000","last_undegraded":"2023-11-17T00:07:02.905512+0000","last_fullsized":"2023-11-17T00:07:02.905512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:34:05.676105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825096+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:07:02.825096+0000","last_peered":"2023-11-17T00:07:02.825096+0000","last_clean":"2023-11-17T00:07:02.825096+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:07:02.825096+0000","last_undegraded":"2023-11-17T00:07:02.825096+0000","last_fullsized":"2023-11-17T00:07:02.825096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:47.945629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823108+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:07:02.823108+0000","last_peered":"2023-11-17T00:07:02.823108+0000","last_clean":"2023-11-17T00:07:02.823108+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:07:02.823108+0000","last_undegraded":"2023-11-17T00:07:02.823108+0000","last_fullsized":"2023-11-17T00:07:02.823108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:30.496908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904122+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:07:02.904122+0000","last_peered":"2023-11-17T00:07:02.904122+0000","last_clean":"2023-11-17T00:07:02.904122+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:07:02.904122+0000","last_undegraded":"2023-11-17T00:07:02.904122+0000","last_fullsized":"2023-11-17T00:07:02.904122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:10:06.214442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827858+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:07:02.827858+0000","last_peered":"2023-11-17T00:07:02.827858+0000","last_clean":"2023-11-17T00:07:02.827858+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:07:02.827858+0000","last_undegraded":"2023-11-17T00:07:02.827858+0000","last_fullsized":"2023-11-17T00:07:02.827858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:44:49.714370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823144+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:07:02.823144+0000","last_peered":"2023-11-17T00:07:02.823144+0000","last_clean":"2023-11-17T00:07:02.823144+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:07:02.823144+0000","last_undegraded":"2023-11-17T00:07:02.823144+0000","last_fullsized":"2023-11-17T00:07:02.823144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:43:01.046315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906790+0000","last_change":"2023-11-17T00:07:02.906790+0000","last_active":"2023-11-17T00:07:02.906790+0000","last_peered":"2023-11-17T00:07:02.906790+0000","last_clean":"2023-11-17T00:07:02.906790+0000","last_became_active":"2023-11-17T00:07:02.906439+0000","last_became_peered":"2023-11-17T00:07:02.906439+0000","last_unstale":"2023-11-17T00:07:02.906790+0000","last_undegraded":"2023-11-17T00:07:02.906790+0000","last_fullsized":"2023-11-17T00:07:02.906790+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:53:03.827866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823276+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:07:02.823276+0000","last_peered":"2023-11-17T00:07:02.823276+0000","last_clean":"2023-11-17T00:07:02.823276+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:07:02.823276+0000","last_undegraded":"2023-11-17T00:07:02.823276+0000","last_fullsized":"2023-11-17T00:07:02.823276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:36:34.182907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822997+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:07:02.822997+0000","last_peered":"2023-11-17T00:07:02.822997+0000","last_clean":"2023-11-17T00:07:02.822997+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:07:02.822997+0000","last_undegraded":"2023-11-17T00:07:02.822997+0000","last_fullsized":"2023-11-17T00:07:02.822997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:24:25.194919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.906389+0000","last_change":"2023-11-17T00:07:02.906389+0000","last_active":"2023-11-17T00:07:02.906389+0000","last_peered":"2023-11-17T00:07:02.906389+0000","last_clean":"2023-11-17T00:07:02.906389+0000","last_became_active":"2023-11-17T00:07:02.905885+0000","last_became_peered":"2023-11-17T00:07:02.905885+0000","last_unstale":"2023-11-17T00:07:02.906389+0000","last_undegraded":"2023-11-17T00:07:02.906389+0000","last_fullsized":"2023-11-17T00:07:02.906389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:01:26.319051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:03.431743+0000","last_change":"2023-11-17T00:07:03.431743+0000","last_active":"2023-11-17T00:07:03.431743+0000","last_peered":"2023-11-17T00:07:03.431743+0000","last_clean":"2023-11-17T00:07:03.431743+0000","last_became_active":"2023-11-17T00:07:02.907841+0000","last_became_peered":"2023-11-17T00:07:02.907841+0000","last_unstale":"2023-11-17T00:07:03.431743+0000","last_undegraded":"2023-11-17T00:07:03.431743+0000","last_fullsized":"2023-11-17T00:07:03.431743+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:49:13.936360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.823548+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:07:02.823548+0000","last_peered":"2023-11-17T00:07:02.823548+0000","last_clean":"2023-11-17T00:07:02.823548+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:07:02.823548+0000","last_undegraded":"2023-11-17T00:07:02.823548+0000","last_fullsized":"2023-11-17T00:07:02.823548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:50:11.202413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825795+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:07:02.825795+0000","last_peered":"2023-11-17T00:07:02.825795+0000","last_clean":"2023-11-17T00:07:02.825795+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:07:02.825795+0000","last_undegraded":"2023-11-17T00:07:02.825795+0000","last_fullsized":"2023-11-17T00:07:02.825795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:13.331240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.832453+0000","last_change":"2023-11-17T00:07:02.832453+0000","last_active":"2023-11-17T00:07:02.832453+0000","last_peered":"2023-11-17T00:07:02.832453+0000","last_clean":"2023-11-17T00:07:02.832453+0000","last_became_active":"2023-11-17T00:07:02.831920+0000","last_became_peered":"2023-11-17T00:07:02.831920+0000","last_unstale":"2023-11-17T00:07:02.832453+0000","last_undegraded":"2023-11-17T00:07:02.832453+0000","last_fullsized":"2023-11-17T00:07:02.832453+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:41.228707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825330+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:07:02.825330+0000","last_peered":"2023-11-17T00:07:02.825330+0000","last_clean":"2023-11-17T00:07:02.825330+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:07:02.825330+0000","last_undegraded":"2023-11-17T00:07:02.825330+0000","last_fullsized":"2023-11-17T00:07:02.825330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:45.221009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825749+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:07:02.825749+0000","last_peered":"2023-11-17T00:07:02.825749+0000","last_clean":"2023-11-17T00:07:02.825749+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:07:02.825749+0000","last_undegraded":"2023-11-17T00:07:02.825749+0000","last_fullsized":"2023-11-17T00:07:02.825749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:18:47.931904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822557+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:07:02.822557+0000","last_peered":"2023-11-17T00:07:02.822557+0000","last_clean":"2023-11-17T00:07:02.822557+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:07:02.822557+0000","last_undegraded":"2023-11-17T00:07:02.822557+0000","last_fullsized":"2023-11-17T00:07:02.822557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:53:58.808751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826173+0000","last_change":"2023-11-17T00:06:12.638454+0000","last_active":"2023-11-17T00:07:02.826173+0000","last_peered":"2023-11-17T00:07:02.826173+0000","last_clean":"2023-11-17T00:07:02.826173+0000","last_became_active":"2023-11-17T00:06:12.637937+0000","last_became_peered":"2023-11-17T00:06:12.637937+0000","last_unstale":"2023-11-17T00:07:02.826173+0000","last_undegraded":"2023-11-17T00:07:02.826173+0000","last_fullsized":"2023-11-17T00:07:02.826173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:31:28.638284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.828066+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:07:02.828066+0000","last_peered":"2023-11-17T00:07:02.828066+0000","last_clean":"2023-11-17T00:07:02.828066+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:07:02.828066+0000","last_undegraded":"2023-11-17T00:07:02.828066+0000","last_fullsized":"2023-11-17T00:07:02.828066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:04:22.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.827817+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:07:02.827817+0000","last_peered":"2023-11-17T00:07:02.827817+0000","last_clean":"2023-11-17T00:07:02.827817+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:07:02.827817+0000","last_undegraded":"2023-11-17T00:07:02.827817+0000","last_fullsized":"2023-11-17T00:07:02.827817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:16:10.946473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822651+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:07:02.822651+0000","last_peered":"2023-11-17T00:07:02.822651+0000","last_clean":"2023-11-17T00:07:02.822651+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:07:02.822651+0000","last_undegraded":"2023-11-17T00:07:02.822651+0000","last_fullsized":"2023-11-17T00:07:02.822651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:42.813047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.825815+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:07:02.825815+0000","last_peered":"2023-11-17T00:07:02.825815+0000","last_clean":"2023-11-17T00:07:02.825815+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:07:02.825815+0000","last_undegraded":"2023-11-17T00:07:02.825815+0000","last_fullsized":"2023-11-17T00:07:02.825815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:42:26.307090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.904459+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:07:02.904459+0000","last_peered":"2023-11-17T00:07:02.904459+0000","last_clean":"2023-11-17T00:07:02.904459+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:07:02.904459+0000","last_undegraded":"2023-11-17T00:07:02.904459+0000","last_fullsized":"2023-11-17T00:07:02.904459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:27:34.972172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.822134+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:07:02.822134+0000","last_peered":"2023-11-17T00:07:02.822134+0000","last_clean":"2023-11-17T00:07:02.822134+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:07:02.822134+0000","last_undegraded":"2023-11-17T00:07:02.822134+0000","last_fullsized":"2023-11-17T00:07:02.822134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:35:22.231740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-17T00:07:02.826314+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:07:02.826314+0000","last_peered":"2023-11-17T00:07:02.826314+0000","last_clean":"2023-11-17T00:07:02.826314+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:07:02.826314+0000","last_undegraded":"2023-11-17T00:07:02.826314+0000","last_fullsized":"2023-11-17T00:07:02.826314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:13:46.566932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62220,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":127,"num_read_kb":126,"num_write":72,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6152,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739000,"statfs":{"total":95995035648,"available":95988736000,"internally_reserved":0,"allocated":983040,"data_stored":494164,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92700000000000005,"5min":1.042,"15min":1.042},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.624,"15min":0.61099999999999999},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63400000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.38,"5min":0.245,"15min":0.245},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50800000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.438,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83899999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97099999999999997,"5min":1.153,"15min":1.153},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.59799999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.95999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60399999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.61199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.476,"15min":1.476},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77300000000000002},"min":{"1min":0.64200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.012,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.89500000000000002}]},{"osd":10,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.77500000000000002},"min":{"1min":0.63900000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0289999999999999,"5min":1.429,"15min":1.429},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.65000000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":1.522,"15min":1.522},"last":0.76500000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":1101824,"data_stored":1141687,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47099999999999997,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66300000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.496,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":0.872},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82499999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.495},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.874,"15min":0.874},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.375},{"interface":"front","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.436}]},{"osd":5,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.496,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85599999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.621}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.747,"15min":0.75},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93600000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":0.996},"last":0.65900000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96099999999999997,"5min":1.115,"15min":1.115},"last":0.63400000000000001}]},{"osd":11,"last update":"Fri Nov 17 00:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75800000000000001,"15min":0.75600000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.753}]}]},{"osd":9,"up_from":66,"seq":283467841574,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6284,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738868,"statfs":{"total":95995035648,"available":95988600832,"internally_reserved":0,"allocated":987136,"data_stored":498808,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64400000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83099999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78900000000000003,"15min":0.80600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84799999999999998,"5min":1.093,"15min":1.093},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.77600000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.224,"15min":1.224},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93999999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93400000000000005,"5min":1.28,"15min":1.28},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.85199999999999998,"15min":0.87},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98999999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.93100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0649999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.92200000000000004,"15min":0.95399999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0900000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.747}]},{"osd":8,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.98299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":1.004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.01,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95499999999999996,"15min":0.97599999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3660000000000001,"5min":1.51,"15min":1.51},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.83199999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6412,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738740,"statfs":{"total":95995035648,"available":95988469760,"internally_reserved":0,"allocated":987136,"data_stored":494734,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81100000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.376},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70299999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.44},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75600000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.55100000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88500000000000001,"5min":1.139,"15min":1.139},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.44700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85299999999999998,"5min":1.075,"15min":1.075},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.60999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.624,"5min":0.503,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Nov 17 00:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.98399999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.877},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Nov 17 00:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.79900000000000004,"15min":0.78500000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.64400000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78700000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.748},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.73799999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6588,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738564,"statfs":{"total":95995035648,"available":95988289536,"internally_reserved":0,"allocated":1036288,"data_stored":1089176,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64000000000000001,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.432,"15min":0.42899999999999999},"min":{"1min":0.29199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.57299999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.78600000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.65000000000000002,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80700000000000005,"5min":1.083,"15min":1.083},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.54000000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71099999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.49299999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.90900000000000003,"5min":1.046,"15min":1.046},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.69899999999999995},"min":{"1min":0.51100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":1.173,"15min":1.173},"last":0.622}]},{"osd":6,"last update":"Fri Nov 17 00:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.73199999999999998,"5min":1.129,"15min":1.129},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.754}]},{"osd":8,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.52600000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.879,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":0.84799999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:06:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91000000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.73299999999999998,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81399999999999995}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.85799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6400,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738752,"statfs":{"total":95995035648,"available":95988482048,"internally_reserved":0,"allocated":974848,"data_stored":490034,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.83599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.623},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.224,"15min":1.224},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68899999999999995,"15min":0.66100000000000003},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.0669999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1679999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.262,"15min":1.262},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.331,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.66900000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79700000000000004,"15min":0.754},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0509999999999999,"5min":1.544,"15min":1.544},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.76200000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.377,"15min":1.377},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.63100000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.472,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0429999999999999}]},{"osd":9,"last update":"Fri Nov 17 00:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.93999999999999995,"15min":0.96299999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.147,"5min":1.429,"15min":1.429},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.96599999999999997,"15min":0.99099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0629999999999999}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1,"15min":1.0169999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.4259999999999999,"5min":1.698,"15min":1.698},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.98299999999999998,"15min":0.99299999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3939999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0249999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7492,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737660,"statfs":{"total":95995035648,"available":95987363840,"internally_reserved":0,"allocated":1044480,"data_stored":551456,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42599999999999999,"5min":0.30499999999999999,"15min":0.27100000000000002},"max":{"1min":0.89600000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.502},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74399999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Nov 17 00:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Nov 17 00:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59299999999999997,"15min":0.54700000000000004},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.63700000000000001,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.626}]},{"osd":4,"last update":"Fri Nov 17 00:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.46200000000000002,"15min":0.41499999999999998},"min":{"1min":0.378,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98199999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.41299999999999998},"min":{"1min":0.28699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.83099999999999996,"5min":1.365,"15min":1.365},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 17 00:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59199999999999997,"15min":0.55800000000000005},"min":{"1min":0.53600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90900000000000003,"5min":1.395,"15min":1.395},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":1.123,"15min":1.123},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.65100000000000002,"15min":0.63300000000000001},"min":{"1min":0.54700000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.53900000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.752},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.95399999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Nov 17 00:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.879},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68400000000000005}]},{"osd":11,"last update":"Fri Nov 17 00:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7364,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737788,"statfs":{"total":95995035648,"available":95987494912,"internally_reserved":0,"allocated":978944,"data_stored":490124,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63300000000000001,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.56999999999999995},"min":{"1min":0.54200000000000004,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.627}]},{"osd":2,"last update":"Fri Nov 17 00:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.59499999999999997},"min":{"1min":0.54100000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.63600000000000001,"15min":0.58999999999999997},"min":{"1min":0.48999999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63900000000000001,"15min":0.58199999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0660000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66500000000000004,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.98999999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.48199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.64100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.749}]},{"osd":8,"last update":"Fri Nov 17 00:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0409999999999999,"5min":1.377,"15min":1.377},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.84799999999999998,"15min":0.81299999999999994},"min":{"1min":0.71899999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1619999999999999,"5min":1.292,"15min":1.292},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.872,"15min":0.87},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0680000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92100000000000004,"15min":0.92500000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.147,"5min":1.194,"15min":1.194},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.204,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.93999999999999995},"min":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:07:24 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":0.98599999999999999,"15min":0.96199999999999997},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.343,"5min":4.343,"15min":4.343},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.78400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7492,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737660,"statfs":{"total":95995035648,"available":95987363840,"internally_reserved":0,"allocated":1044480,"data_stored":551456,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53700000000000003,"15min":0.48499999999999999},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.42899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60699999999999998,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91400000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85499999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42799999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.629}]},{"osd":5,"last update":"Fri Nov 17 00:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.48999999999999999,"15min":0.46500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Nov 17 00:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.67300000000000004,"15min":0.628},"min":{"1min":0.60899999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67000000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.83599999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.51200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.58799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Nov 17 00:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.73099999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.59999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.76700000000000002,"15min":0.74299999999999999},"min":{"1min":0.75700000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.92300000000000004}]},{"osd":11,"last update":"Fri Nov 17 00:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.753},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.89900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6724,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738428,"statfs":{"total":95995035648,"available":95988150272,"internally_reserved":0,"allocated":978944,"data_stored":489807,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.41999999999999998},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48999999999999999,"15min":0.44600000000000001},"min":{"1min":0.373,"5min":0.255,"15min":0.255},"max":{"1min":0.62,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.65100000000000002,"15min":0.59799999999999998},"min":{"1min":0.59299999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.60299999999999998},"min":{"1min":0.54400000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94399999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69199999999999995,"15min":0.64300000000000002},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95499999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92800000000000005,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.66700000000000004},"min":{"1min":0.55900000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80900000000000005,"5min":1.232,"15min":1.232},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":1.206,"15min":1.206},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Nov 17 00:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.67800000000000005},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.357,"5min":1.359,"15min":1.359},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.755},"min":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2789999999999999,"5min":1.46,"15min":1.46},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.753,"15min":0.71699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.288,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.90000000000000002,"15min":0.92600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.004,"15min":1.034},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2030000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.504}]},{"osd":10,"last update":"Fri Nov 17 00:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.98799999999999999,"15min":1.012},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.98999999999999999,"5min":1.496,"15min":1.496},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.012,"15min":1.028},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Fri Nov 17 00:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.79400000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6600,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738552,"statfs":{"total":95995035648,"available":95988277248,"internally_reserved":0,"allocated":983040,"data_stored":494152,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.35199999999999998},"min":{"1min":0.314,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.41599999999999998,"15min":0.36699999999999999},"min":{"1min":0.32900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61699999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.52000000000000002},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.875,"15min":0.875},"last":0.435},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.50700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61599999999999999,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":5,"last update":"Fri Nov 17 00:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.56799999999999995},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84799999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.57499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63}]},{"osd":6,"last update":"Fri Nov 17 00:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66700000000000004,"15min":0.64500000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82199999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.83199999999999996}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.52900000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72699999999999998,"15min":0.70499999999999996},"min":{"1min":0.57399999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Nov 17 00:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80000000000000004,"15min":0.80700000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96699999999999997,"5min":1.081,"15min":1.081},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.54500000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0169999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.505,"5min":0.436,"15min":0.436},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56200000000000006}]},{"osd":11,"last update":"Fri Nov 17 00:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67100000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6720,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738432,"statfs":{"total":95995035648,"available":95988154368,"internally_reserved":0,"allocated":1040384,"data_stored":1084680,"data_compressed":53432,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53000000000000003},"min":{"1min":0.45500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.86599999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.46200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.41499999999999998,"15min":0.38800000000000001},"min":{"1min":0.36899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.42899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.752}]},{"osd":4,"last update":"Fri Nov 17 00:06:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61599999999999999,"15min":0.57899999999999996},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84199999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Nov 17 00:07:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83899999999999997,"5min":1.05,"15min":1.05},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.60499999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.046,"15min":1.046},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.50600000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91000000000000003}]},{"osd":8,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.46999999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85799999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.755,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.93300000000000005}]},{"osd":10,"last update":"Fri Nov 17 00:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95399999999999996,"5min":1.073,"15min":1.073},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:07:36.420 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-17T00:07:36.420 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-17T00:07:36.420 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-17T00:07:36.420 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph health --format=json 2023-11-17T00:07:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:36 smithi186 ceph-mon[131487]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:36 smithi186 ceph-mon[131487]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:36 smithi006 ceph-mon[97840]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:37.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:36 smithi006 ceph-mon[97840]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:37.285 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:37.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:36 smithi087 ceph-mon[131182]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:07:37.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:36 smithi087 ceph-mon[131182]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:39.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:38 smithi006 ceph-mon[97840]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 -- 172.21.15.6:0/326623024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c100ee0 msgr2=0x7f809c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 --2- 172.21.15.6:0/326623024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c100ee0 0x7f809c104eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f808c009990 tx=0x7f808c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 -- 172.21.15.6:0/326623024 shutdown_connections 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 --2- 172.21.15.6:0/326623024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c1055e0 0x7f809c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 --2- 172.21.15.6:0/326623024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 0x7f809c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 --2- 172.21.15.6:0/326623024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c100ee0 0x7f809c104eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 -- 172.21.15.6:0/326623024 >> 172.21.15.6:0/326623024 conn(0x7f809c0fbdc0 msgr2=0x7f809c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:39.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 -- 172.21.15.6:0/326623024 shutdown_connections 2023-11-17T00:07:39.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.309+0000 7f80a47b2700 1 -- 172.21.15.6:0/326623024 wait complete. 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.310+0000 7f80a47b2700 1 Processor -- start 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.310+0000 7f80a47b2700 1 -- start start 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 0x7f809c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c1055e0 0x7f809c073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a4850 con 0x7f809c100ee0 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a49c0 con 0x7f809c1055e0 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a47b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a4b80 con 0x7f809c100540 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a1d4d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a1d4d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35904/0 (socket says 172.21.15.6:35904) 2023-11-17T00:07:39.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a1d4d700 1 -- 172.21.15.6:0/3810979097 learned_addr learned my addr 172.21.15.6:0/3810979097 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:39.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a254e700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 0x7f809c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:39.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.311+0000 7f80a2d4f700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c1055e0 0x7f809c073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:39.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 -- 172.21.15.6:0/3810979097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 msgr2=0x7f809c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 0x7f809c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 -- 172.21.15.6:0/3810979097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c1055e0 msgr2=0x7f809c073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c1055e0 0x7f809c073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 -- 172.21.15.6:0/3810979097 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009c40 con 0x7f809c100ee0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a1d4d700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f808c004740 tx=0x7f808c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c01d070 con 0x7f809c100ee0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808c004830 con 0x7f809c100ee0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c0095f0 con 0x7f809c100ee0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0052c0 con 0x7f809c100ee0 2023-11-17T00:07:39.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.312+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c110360 con 0x7f809c100ee0 2023-11-17T00:07:39.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.313+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c1101d0 con 0x7f809c100ee0 2023-11-17T00:07:39.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.315+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f808c005480 con 0x7f809c100ee0 2023-11-17T00:07:39.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.317+0000 7f80937fe700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 0x7f80880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:39.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.317+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f808c0cde50 con 0x7f809c100ee0 2023-11-17T00:07:39.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.317+0000 7f80a254e700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 0x7f80880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:39.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.318+0000 7f80a254e700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 0x7f80880a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8098009c60 tx=0x7f8098009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:39.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.321+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f808c05a730 con 0x7f809c100ee0 2023-11-17T00:07:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:38 smithi087 ceph-mon[131182]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:38 smithi186 ceph-mon[131487]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:39.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.577+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f809c1104f0 con 0x7f809c100ee0 2023-11-17T00:07:39.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.577+0000 7f80937fe700 1 -- 172.21.15.6:0/3810979097 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f808c097160 con 0x7f809c100ee0 2023-11-17T00:07:39.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:39.582 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-17T00:07:39.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.579+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 msgr2=0x7f80880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:39.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.579+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 0x7f80880a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8098009c60 tx=0x7f8098009260 comp rx=0 tx=0).stop 2023-11-17T00:07:39.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.579+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 msgr2=0x7f809c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:39.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.579+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f808c004740 tx=0x7f808c009580 comp rx=0 tx=0).stop 2023-11-17T00:07:39.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 shutdown_connections 2023-11-17T00:07:39.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f80880a11e0 0x7f80880a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f809c1055e0 0x7f809c073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f809c100540 0x7f809c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 --2- 172.21.15.6:0/3810979097 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f809c100ee0 0x7f809c073330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:39.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 >> 172.21.15.6:0/3810979097 conn(0x7f809c0fbdc0 msgr2=0x7f809c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:39.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 shutdown_connections 2023-11-17T00:07:39.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:39.580+0000 7f80a47b2700 1 -- 172.21.15.6:0/3810979097 wait complete. 2023-11-17T00:07:40.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:39 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3810979097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-17T00:07:40.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:39 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3810979097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-17T00:07:40.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:39 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3810979097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-17T00:07:40.393 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-17T00:07:40.393 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:07:40.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 -- 172.21.15.6:0/265978944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1002d0 msgr2=0x7fa96c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 --2- 172.21.15.6:0/265978944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1002d0 0x7fa96c1006e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009990 tx=0x7fa95c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 -- 172.21.15.6:0/265978944 shutdown_connections 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 --2- 172.21.15.6:0/265978944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1002d0 0x7fa96c1006e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 --2- 172.21.15.6:0/265978944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c101e50 0x7fa96c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 --2- 172.21.15.6:0/265978944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c1014c0 0x7fa96c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 -- 172.21.15.6:0/265978944 >> 172.21.15.6:0/265978944 conn(0x7fa96c0fb8b0 msgr2=0x7fa96c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 -- 172.21.15.6:0/265978944 shutdown_connections 2023-11-17T00:07:40.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.602+0000 7fa9711ff700 1 -- 172.21.15.6:0/265978944 wait complete. 2023-11-17T00:07:40.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 Processor -- start 2023-11-17T00:07:40.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 -- start start 2023-11-17T00:07:40.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:40.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c101e50 0x7fa96c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c19fde0 con 0x7fa96c1014c0 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c19ff20 con 0x7fa96c1002d0 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.603+0000 7fa9711ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c1a00b0 con 0x7fa96c101e50 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35928/0 (socket says 172.21.15.6:35928) 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96a59c700 1 -- 172.21.15.6:0/1933139215 learned_addr learned my addr 172.21.15.6:0/1933139215 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60012/0 (socket says 172.21.15.6:60012) 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96b59e700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c101e50 0x7fa96c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 -- 172.21.15.6:0/1933139215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c101e50 msgr2=0x7fa96c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:40.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c101e50 0x7fa96c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 -- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 msgr2=0x7fa96c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:40.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 -- 172.21.15.6:0/1933139215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c0095f0 con 0x7fa96c1002d0 2023-11-17T00:07:40.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96a59c700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:40.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.604+0000 7fa96ad9d700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00b620 tx=0x7fa95c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:40.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.605+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c01d070 con 0x7fa96c1002d0 2023-11-17T00:07:40.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.605+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c10bca0 con 0x7fa96c1002d0 2023-11-17T00:07:40.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.605+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95c021410 con 0x7fa96c1002d0 2023-11-17T00:07:40.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.605+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c017650 con 0x7fa96c1002d0 2023-11-17T00:07:40.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.605+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c10bb10 con 0x7fa96c1002d0 2023-11-17T00:07:40.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.606+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c1a0650 con 0x7fa96c1002d0 2023-11-17T00:07:40.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.608+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa95c0217f0 con 0x7fa96c1002d0 2023-11-17T00:07:40.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.609+0000 7fa953fff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 0x7fa9540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:40.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.609+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa95c0ce710 con 0x7fa96c1002d0 2023-11-17T00:07:40.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.610+0000 7fa96a59c700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 0x7fa9540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:40.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.610+0000 7fa96a59c700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 0x7fa9540a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa960011550 tx=0x7fa960009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:40.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.611+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa95c097a20 con 0x7fa96c1002d0 2023-11-17T00:07:40.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.829+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa96c1a0c00 con 0x7fa96c1002d0 2023-11-17T00:07:40.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.831+0000 7fa953fff700 1 -- 172.21.15.6:0/1933139215 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7fa95c021e60 con 0x7fa96c1002d0 2023-11-17T00:07:40.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:40.833 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":90,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:07:02.818255+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:07:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.833+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 msgr2=0x7fa9540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.833+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 0x7fa9540a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa960011550 tx=0x7fa960009260 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.833+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 msgr2=0x7fa96c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.833+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00b620 tx=0x7fa95c004730 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 shutdown_connections 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa9540a11e0 0x7fa9540a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa96c101e50 0x7fa96c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa96c1002d0 0x7fa96c198260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 --2- 172.21.15.6:0/1933139215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa96c1014c0 0x7fa96c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 >> 172.21.15.6:0/1933139215 conn(0x7fa96c0fb8b0 msgr2=0x7fa96c106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 shutdown_connections 2023-11-17T00:07:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:40.834+0000 7fa9711ff700 1 -- 172.21.15.6:0/1933139215 wait complete. 2023-11-17T00:07:40.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:07:40.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:07:41.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.065+0000 7f457c3ff700 1 -- 172.21.15.6:0/4058563620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574101480 msgr2=0x7f45741018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.065+0000 7f457c3ff700 1 --2- 172.21.15.6:0/4058563620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574101480 0x7f45741018d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4564009990 tx=0x7f4564009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 -- 172.21.15.6:0/4058563620 shutdown_connections 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 --2- 172.21.15.6:0/4058563620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101e10 0x7f45741086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 --2- 172.21.15.6:0/4058563620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574100290 0x7f45741006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 --2- 172.21.15.6:0/4058563620 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574101480 0x7f45741018d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 -- 172.21.15.6:0/4058563620 >> 172.21.15.6:0/4058563620 conn(0x7f45740fb870 msgr2=0x7f45740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 -- 172.21.15.6:0/4058563620 shutdown_connections 2023-11-17T00:07:41.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.066+0000 7f457c3ff700 1 -- 172.21.15.6:0/4058563620 wait complete. 2023-11-17T00:07:41.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 Processor -- start 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 -- start start 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574101e10 0x7f457419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419db80 con 0x7f4574100290 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419dcc0 con 0x7f4574101480 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.067+0000 7f457c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419de50 con 0x7f4574101e10 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35950/0 (socket says 172.21.15.6:35950) 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 -- 172.21.15.6:0/1394694529 learned_addr learned my addr 172.21.15.6:0/1394694529 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:41.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60036/0 (socket says 172.21.15.6:60036) 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a99c700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574101e10 0x7f457419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 -- 172.21.15.6:0/1394694529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574101e10 msgr2=0x7f457419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574101e10 0x7f457419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 -- 172.21.15.6:0/1394694529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 msgr2=0x7f4574196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 -- 172.21.15.6:0/1394694529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640095f0 con 0x7f4574100290 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457999a700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f457a19b700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f457000efc0 tx=0x7f457000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.068+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570018070 con 0x7f4574100290 2023-11-17T00:07:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.069+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f457000f040 con 0x7f4574100290 2023-11-17T00:07:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.069+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457410bc50 con 0x7f4574100290 2023-11-17T00:07:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.069+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570004280 con 0x7f4574100290 2023-11-17T00:07:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.069+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457410bac0 con 0x7f4574100290 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.070+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4574107870 con 0x7f4574100290 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.072+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4570003a00 con 0x7f4574100290 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.073+0000 7f456b7fe700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 0x7f45600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.073+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45700ccc80 con 0x7f4574100290 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.073+0000 7f457999a700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 0x7f45600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.074+0000 7f457999a700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 0x7f45600a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f456400b2e0 tx=0x7f4564005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.077+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4570095ff0 con 0x7f4574100290 2023-11-17T00:07:41.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:40 smithi006 ceph-mon[97840]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:40 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1933139215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:41.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.284+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4574067a50 con 0x7f4574100290 2023-11-17T00:07:41.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.284+0000 7f456b7fe700 1 -- 172.21.15.6:0/1394694529 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4570099610 con 0x7f4574100290 2023-11-17T00:07:41.286 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:07:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 msgr2=0x7f45600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 0x7f45600a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f456400b2e0 tx=0x7f4564005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 msgr2=0x7f4574196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f457000efc0 tx=0x7f457000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 shutdown_connections 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f45600a11e0 0x7f45600a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4574100290 0x7f4574196000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4574101e10 0x7f457419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 --2- 172.21.15.6:0/1394694529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4574101480 0x7f4574196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 >> 172.21.15.6:0/1394694529 conn(0x7f45740fb870 msgr2=0x7f45741049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 shutdown_connections 2023-11-17T00:07:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.287+0000 7f457c3ff700 1 -- 172.21.15.6:0/1394694529 wait complete. 2023-11-17T00:07:41.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:07:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:40 smithi087 ceph-mon[131182]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:40 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1933139215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:40 smithi186 ceph-mon[131487]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:40 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1933139215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:41.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 -- 172.21.15.6:0/2485874884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0100290 msgr2=0x7fc8f01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/2485874884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0100290 0x7fc8f01006a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009990 tx=0x7fc8e0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 -- 172.21.15.6:0/2485874884 shutdown_connections 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/2485874884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0100290 0x7fc8f01006a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/2485874884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/2485874884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0101480 0x7fc8f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 -- 172.21.15.6:0/2485874884 >> 172.21.15.6:0/2485874884 conn(0x7fc8f00fb870 msgr2=0x7fc8f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 -- 172.21.15.6:0/2485874884 shutdown_connections 2023-11-17T00:07:41.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.533+0000 7fc8f51ff700 1 -- 172.21.15.6:0/2485874884 wait complete. 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.534+0000 7fc8f51ff700 1 Processor -- start 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 -- start start 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f019fe20 con 0x7fc8f0101480 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f019ff60 con 0x7fc8f0100290 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8f51ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f01a00f0 con 0x7fc8f0101e10 2023-11-17T00:07:41.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8ee59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8eed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8eed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60052/0 (socket says 172.21.15.6:60052) 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8ee59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35978/0 (socket says 172.21.15.6:35978) 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8eed9d700 1 -- 172.21.15.6:0/274295759 learned_addr learned my addr 172.21.15.6:0/274295759 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.535+0000 7fc8ef59e700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 -- 172.21.15.6:0/274295759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 msgr2=0x7fc8f019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 -- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 msgr2=0x7fc8f01987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 -- 172.21.15.6:0/274295759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00095f0 con 0x7fc8f0100290 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8ef59e700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8ee59c700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:41.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8eed9d700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005740 tx=0x7fc8e0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e001d070 con 0x7fc8f0100290 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f010bc40 con 0x7fc8f0100290 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8e0021410 con 0x7fc8f0100290 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0017650 con 0x7fc8f0100290 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.536+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f010bab0 con 0x7fc8f0100290 2023-11-17T00:07:41.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.537+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f01928d0 con 0x7fc8f0100290 2023-11-17T00:07:41.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.540+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc8e0021580 con 0x7fc8f0100290 2023-11-17T00:07:41.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.541+0000 7fc8d7fff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 0x7fc8d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.541+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8e00ce8a0 con 0x7fc8f0100290 2023-11-17T00:07:41.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.541+0000 7fc8ee59c700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 0x7fc8d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.542+0000 7fc8ee59c700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 0x7fc8d80a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011490 tx=0x7fc8e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.543+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc8e0097c40 con 0x7fc8f0100290 2023-11-17T00:07:41.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.754+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8f0061700 con 0x7fc8f0100290 2023-11-17T00:07:41.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.755+0000 7fc8d7fff700 1 -- 172.21.15.6:0/274295759 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8e009b260 con 0x7fc8f0100290 2023-11-17T00:07:41.756 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 msgr2=0x7fc8d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 0x7fc8d80a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011490 tx=0x7fc8e4009260 comp rx=0 tx=0).stop 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 msgr2=0x7fc8f01982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005740 tx=0x7fc8e0005770 comp rx=0 tx=0).stop 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 shutdown_connections 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc8d80a11e0 0x7fc8d80a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8f0101e10 0x7fc8f019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8f0100290 0x7fc8f01982a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 --2- 172.21.15.6:0/274295759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8f0101480 0x7fc8f01987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 >> 172.21.15.6:0/274295759 conn(0x7fc8f00fb870 msgr2=0x7fc8f0106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:41.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 shutdown_connections 2023-11-17T00:07:41.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.757+0000 7fc8f51ff700 1 -- 172.21.15.6:0/274295759 wait complete. 2023-11-17T00:07:41.778 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:07:41.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 -- 172.21.15.6:0/4186322275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50073550 msgr2=0x7fbe500739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 --2- 172.21.15.6:0/4186322275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50073550 0x7fbe500739c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009990 tx=0x7fbe40009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 -- 172.21.15.6:0/4186322275 shutdown_connections 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 --2- 172.21.15.6:0/4186322275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 0x7fbe5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 --2- 172.21.15.6:0/4186322275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50074b30 0x7fbe50072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 --2- 172.21.15.6:0/4186322275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50073550 0x7fbe500739c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 -- 172.21.15.6:0/4186322275 >> 172.21.15.6:0/4186322275 conn(0x7fbe500fb870 msgr2=0x7fbe500fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 -- 172.21.15.6:0/4186322275 shutdown_connections 2023-11-17T00:07:41.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.985+0000 7fbe575ff700 1 -- 172.21.15.6:0/4186322275 wait complete. 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.986+0000 7fbe575ff700 1 Processor -- start 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 -- start start 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50074b30 0x7fbe5019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 0x7fbe501a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501a41b0 con 0x7fbe50074b30 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501a42f0 con 0x7fbe50073550 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe575ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501a4480 con 0x7fbe5010c860 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe54b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50074b30 0x7fbe5019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe5539b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe5539b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60080/0 (socket says 172.21.15.6:60080) 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe5539b700 1 -- 172.21.15.6:0/743793055 learned_addr learned my addr 172.21.15.6:0/743793055 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.987+0000 7fbe55b9c700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 0x7fbe501a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 -- 172.21.15.6:0/743793055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 msgr2=0x7fbe501a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 0x7fbe501a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 -- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50074b30 msgr2=0x7fbe5019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50074b30 0x7fbe5019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 -- 172.21.15.6:0/743793055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe400095f0 con 0x7fbe50073550 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe5539b700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e970 tx=0x7fbe4c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009790 con 0x7fbe50073550 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe50112270 con 0x7fbe50073550 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c00c7e0 con 0x7fbe50073550 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0054e0 con 0x7fbe50073550 2023-11-17T00:07:41.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.988+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501120e0 con 0x7fbe50073550 2023-11-17T00:07:41.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.989+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe501023b0 con 0x7fbe50073550 2023-11-17T00:07:41.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.991+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fbe4c00c950 con 0x7fbe50073550 2023-11-17T00:07:41.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.993+0000 7fbe467fc700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 0x7fbe3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:41.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.993+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbe4c014070 con 0x7fbe50073550 2023-11-17T00:07:41.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.993+0000 7fbe54b9a700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 0x7fbe3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:41.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.993+0000 7fbe54b9a700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 0x7fbe3c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b2e0 tx=0x7fbe40005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:41.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:41.995+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe4c096990 con 0x7fbe50073550 2023-11-17T00:07:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.201+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe50067a50 con 0x7fbe50073550 2023-11-17T00:07:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.202+0000 7fbe467fc700 1 -- 172.21.15.6:0/743793055 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe4c099fb0 con 0x7fbe50073550 2023-11-17T00:07:42.204 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 msgr2=0x7fbe3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 0x7fbe3c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b2e0 tx=0x7fbe40005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 msgr2=0x7fbe5019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e970 tx=0x7fbe4c00ec80 comp rx=0 tx=0).stop 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 shutdown_connections 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe3c0a11e0 0x7fbe3c0a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe5010c860 0x7fbe501a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe50073550 0x7fbe5019c630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 --2- 172.21.15.6:0/743793055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe50074b30 0x7fbe5019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 >> 172.21.15.6:0/743793055 conn(0x7fbe500fb870 msgr2=0x7fbe500fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:42.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 shutdown_connections 2023-11-17T00:07:42.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.204+0000 7fbe575ff700 1 -- 172.21.15.6:0/743793055 wait complete. 2023-11-17T00:07:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1394694529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/274295759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.226 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:07:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:41 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1394694529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:41 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/274295759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:41 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1394694529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:41 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/274295759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 -- 172.21.15.6:0/4215071730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84100d90 msgr2=0x7efc84103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/4215071730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84100d90 0x7efc84103170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc80009990 tx=0x7efc80009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 -- 172.21.15.6:0/4215071730 shutdown_connections 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/4215071730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84100d90 0x7efc84103170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/4215071730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc84105fd0 0x7efc841083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/4215071730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc841036b0 0x7efc84105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 -- 172.21.15.6:0/4215071730 >> 172.21.15.6:0/4215071730 conn(0x7efc840fa750 msgr2=0x7efc840fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 -- 172.21.15.6:0/4215071730 shutdown_connections 2023-11-17T00:07:42.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.433+0000 7efc8b3ff700 1 -- 172.21.15.6:0/4215071730 wait complete. 2023-11-17T00:07:42.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.434+0000 7efc8b3ff700 1 Processor -- start 2023-11-17T00:07:42.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.434+0000 7efc8b3ff700 1 -- start start 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc841036b0 0x7efc840fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84105fd0 0x7efc840ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840ffa20 con 0x7efc84105fd0 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840ffb90 con 0x7efc84100d90 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841a0080 con 0x7efc841036b0 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8999c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84105fd0 0x7efc840ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8899a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc841036b0 0x7efc840fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60090/0 (socket says 172.21.15.6:60090) 2023-11-17T00:07:42.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 -- 172.21.15.6:0/2909862122 learned_addr learned my addr 172.21.15.6:0/2909862122 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:42.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 -- 172.21.15.6:0/2909862122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc841036b0 msgr2=0x7efc840fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc841036b0 0x7efc840fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 -- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84105fd0 msgr2=0x7efc840ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84105fd0 0x7efc840ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.435+0000 7efc8919b700 1 -- 172.21.15.6:0/2909862122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800095f0 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc8919b700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc800098e0 tx=0x7efc80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8001d070 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc800054f0 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8410bc40 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80004be0 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.436+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8410bab0 con 0x7efc84100d90 2023-11-17T00:07:42.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.437+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc840fc2c0 con 0x7efc84100d90 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.439+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efc8000fbc0 con 0x7efc84100d90 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.440+0000 7efc7a7fc700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 0x7efc700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.441+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc800ce4e0 con 0x7efc84100d90 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.441+0000 7efc8899a700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 0x7efc700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.441+0000 7efc8899a700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 0x7efc700a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc74009c60 tx=0x7efc74009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:42.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.442+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efc800977f0 con 0x7efc84100d90 2023-11-17T00:07:42.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.654+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efc84067a50 con 0x7efc84100d90 2023-11-17T00:07:42.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.655+0000 7efc7a7fc700 1 -- 172.21.15.6:0/2909862122 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7efc8009ae10 con 0x7efc84100d90 2023-11-17T00:07:42.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 msgr2=0x7efc700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 0x7efc700a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc74009c60 tx=0x7efc74009260 comp rx=0 tx=0).stop 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 msgr2=0x7efc84100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc800098e0 tx=0x7efc80004730 comp rx=0 tx=0).stop 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 shutdown_connections 2023-11-17T00:07:42.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efc700a11e0 0x7efc700a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc84105fd0 0x7efc840ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc84100d90 0x7efc84100700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 --2- 172.21.15.6:0/2909862122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc841036b0 0x7efc840fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 >> 172.21.15.6:0/2909862122 conn(0x7efc840fa750 msgr2=0x7efc841019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 shutdown_connections 2023-11-17T00:07:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.657+0000 7efc8b3ff700 1 -- 172.21.15.6:0/2909862122 wait complete. 2023-11-17T00:07:42.679 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:07:42.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.885+0000 7f8b077ff700 1 -- 172.21.15.6:0/2150653739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b00102530 msgr2=0x7f8b00102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.885+0000 7f8b077ff700 1 --2- 172.21.15.6:0/2150653739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b00102530 0x7f8b00102940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009990 tx=0x7f8afc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 -- 172.21.15.6:0/2150653739 shutdown_connections 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 --2- 172.21.15.6:0/2150653739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b001040b0 0x7f8b0010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 --2- 172.21.15.6:0/2150653739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b00102530 0x7f8b00102940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 --2- 172.21.15.6:0/2150653739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00103720 0x7f8b00103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 -- 172.21.15.6:0/2150653739 >> 172.21.15.6:0/2150653739 conn(0x7f8b000fdb30 msgr2=0x7f8b000fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 -- 172.21.15.6:0/2150653739 shutdown_connections 2023-11-17T00:07:42.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.886+0000 7f8b077ff700 1 -- 172.21.15.6:0/2150653739 wait complete. 2023-11-17T00:07:42.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.887+0000 7f8b077ff700 1 Processor -- start 2023-11-17T00:07:42.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.887+0000 7f8b077ff700 1 -- start start 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b00103720 0x7f8b00198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b001040b0 0x7f8b0019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0019fe40 con 0x7f8b00102530 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0019ff80 con 0x7f8b001040b0 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b077ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001a0110 con 0x7f8b00103720 2023-11-17T00:07:42.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b0559b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b0559b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36044/0 (socket says 172.21.15.6:36044) 2023-11-17T00:07:42.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b0559b700 1 -- 172.21.15.6:0/4028329832 learned_addr learned my addr 172.21.15.6:0/4028329832 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:42.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b04d9a700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b00103720 0x7f8b00198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.888+0000 7f8b05d9c700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b001040b0 0x7f8b0019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 -- 172.21.15.6:0/4028329832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b00103720 msgr2=0x7f8b00198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b00103720 0x7f8b00198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 -- 172.21.15.6:0/4028329832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b001040b0 msgr2=0x7f8b0019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b001040b0 0x7f8b0019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 -- 172.21.15.6:0/4028329832 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0095f0 con 0x7f8b00102530 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b0559b700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b620 tx=0x7f8afc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc01d070 con 0x7f8b00102530 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8afc021410 con 0x7f8b00102530 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0010def0 con 0x7f8b00102530 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc017650 con 0x7f8b00102530 2023-11-17T00:07:42.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.889+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0010dd60 con 0x7f8b00102530 2023-11-17T00:07:42.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.891+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b001a0710 con 0x7f8b00102530 2023-11-17T00:07:42.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.893+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8afc021580 con 0x7f8b00102530 2023-11-17T00:07:42.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.894+0000 7f8af67fc700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 0x7f8aec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:42.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.894+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8afc0ce1e0 con 0x7f8b00102530 2023-11-17T00:07:42.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.894+0000 7f8b04d9a700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 0x7f8aec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:42.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.895+0000 7f8b04d9a700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 0x7f8aec0a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009c60 tx=0x7f8af0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:42.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:42.898+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8afc0974f0 con 0x7f8b00102530 2023-11-17T00:07:42.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:42 smithi006 ceph-mon[97840]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:42.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/743793055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:42.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2909862122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:43.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.156+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b001093e0 con 0x7f8b00102530 2023-11-17T00:07:43.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.157+0000 7f8af67fc700 1 -- 172.21.15.6:0/4028329832 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f8afc09ab10 con 0x7f8b00102530 2023-11-17T00:07:43.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:43.159 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:07:43.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 msgr2=0x7f8aec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 0x7f8aec0a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009c60 tx=0x7f8af0009260 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 msgr2=0x7f8b001982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b620 tx=0x7f8afc005740 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 shutdown_connections 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8aec0a1260 0x7f8aec0a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b00102530 0x7f8b001982c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b001040b0 0x7f8b0019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 --2- 172.21.15.6:0/4028329832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b00103720 0x7f8b00198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 >> 172.21.15.6:0/4028329832 conn(0x7f8b000fdb30 msgr2=0x7f8b000ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:43.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 shutdown_connections 2023-11-17T00:07:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.160+0000 7f8b077ff700 1 -- 172.21.15.6:0/4028329832 wait complete. 2023-11-17T00:07:43.164 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:07:43.181 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:07:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:42 smithi087 ceph-mon[131182]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/743793055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2909862122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:42 smithi186 ceph-mon[131487]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:43.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/743793055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:43.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2909862122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 -- 172.21.15.6:0/761970823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f10101e50 msgr2=0x7f8f10108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 --2- 172.21.15.6:0/761970823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f10101e50 0x7f8f10108710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b0a0 tx=0x7f8f0800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 -- 172.21.15.6:0/761970823 shutdown_connections 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 --2- 172.21.15.6:0/761970823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f10101e50 0x7f8f10108710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 --2- 172.21.15.6:0/761970823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101002d0 0x7f8f101006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 --2- 172.21.15.6:0/761970823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f101014c0 0x7f8f10101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.384+0000 7f8f153ff700 1 -- 172.21.15.6:0/761970823 >> 172.21.15.6:0/761970823 conn(0x7f8f100fb8b0 msgr2=0x7f8f100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:43.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.385+0000 7f8f153ff700 1 -- 172.21.15.6:0/761970823 shutdown_connections 2023-11-17T00:07:43.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.385+0000 7f8f153ff700 1 -- 172.21.15.6:0/761970823 wait complete. 2023-11-17T00:07:43.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.385+0000 7f8f153ff700 1 Processor -- start 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 -- start start 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101014c0 0x7f8f10072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 0x7f8f10073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f10073bc0 con 0x7f8f10101e50 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f10073d30 con 0x7f8f101014c0 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f153ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1019ff20 con 0x7f8f101002d0 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f0effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f0f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 0x7f8f10073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f0effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44666/0 (socket says 172.21.15.6:44666) 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f0effd700 1 -- 172.21.15.6:0/988968435 learned_addr learned my addr 172.21.15.6:0/988968435 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.386+0000 7f8f07fff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101014c0 0x7f8f10072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 -- 172.21.15.6:0/988968435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101014c0 msgr2=0x7f8f10072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101014c0 0x7f8f10072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 -- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 msgr2=0x7f8f10073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 0x7f8f10073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 -- 172.21.15.6:0/988968435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef8009520 con 0x7f8f101002d0 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0f7fe700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 0x7f8f10073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0effd700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000e970 tx=0x7f8f0000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:43.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00009790 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f0000c7e0 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00003c70 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f08009d00 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.387+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1010bb80 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.388+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10067af0 con 0x7f8f101002d0 2023-11-17T00:07:43.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.390+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8f0000cc50 con 0x7f8f101002d0 2023-11-17T00:07:43.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.392+0000 7f8f0cff9700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 0x7f8efc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.392+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8f00014070 con 0x7f8f101002d0 2023-11-17T00:07:43.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.392+0000 7f8f07fff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 0x7f8efc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.393+0000 7f8f07fff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 0x7f8efc0a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009df0 tx=0x7f8ef8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:43.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.394+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8f00096820 con 0x7f8f101002d0 2023-11-17T00:07:43.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.601+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f10107110 con 0x7f8f101002d0 2023-11-17T00:07:43.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.604+0000 7f8f0cff9700 1 -- 172.21.15.6:0/988968435 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7f8f00099e40 con 0x7f8f101002d0 2023-11-17T00:07:43.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:43.606 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":90,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:07:02.818255+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 msgr2=0x7f8efc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 0x7f8efc0a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009df0 tx=0x7f8ef8009260 comp rx=0 tx=0).stop 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 msgr2=0x7f8f100748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000e970 tx=0x7f8f0000ec80 comp rx=0 tx=0).stop 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 shutdown_connections 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f8efc0a1260 0x7f8efc0a3710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8f10101e50 0x7f8f10073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f101002d0 0x7f8f100748a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 --2- 172.21.15.6:0/988968435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f101014c0 0x7f8f10072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 >> 172.21.15.6:0/988968435 conn(0x7f8f100fb8b0 msgr2=0x7f8f100fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:43.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 shutdown_connections 2023-11-17T00:07:43.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.606+0000 7f8f153ff700 1 -- 172.21.15.6:0/988968435 wait complete. 2023-11-17T00:07:43.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:07:43.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 -- 172.21.15.6:0/90633261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 msgr2=0x7f070c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 --2- 172.21.15.6:0/90633261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c1006b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009990 tx=0x7f06fc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 -- 172.21.15.6:0/90633261 shutdown_connections 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 --2- 172.21.15.6:0/90633261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c1006b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 --2- 172.21.15.6:0/90633261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 --2- 172.21.15.6:0/90633261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 0x7f070c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 -- 172.21.15.6:0/90633261 >> 172.21.15.6:0/90633261 conn(0x7f070c0fb880 msgr2=0x7f070c0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 -- 172.21.15.6:0/90633261 shutdown_connections 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.836+0000 7f07123ff700 1 -- 172.21.15.6:0/90633261 wait complete. 2023-11-17T00:07:43.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.837+0000 7f07123ff700 1 Processor -- start 2023-11-17T00:07:43.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.837+0000 7f07123ff700 1 -- start start 2023-11-17T00:07:43.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 0x7f070c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19dbb0 con 0x7f070c1002a0 2023-11-17T00:07:43.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19dcf0 con 0x7f070c101490 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f07123ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c19de80 con 0x7f070c101e20 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36064/0 (socket says 172.21.15.6:36064) 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 -- 172.21.15.6:0/1720170973 learned_addr learned my addr 172.21.15.6:0/1720170973 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f0703fff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 0x7f070c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f071099c700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 -- 172.21.15.6:0/1720170973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 msgr2=0x7f070c19d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c19d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 -- 172.21.15.6:0/1720170973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 msgr2=0x7f070c196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:43.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 0x7f070c196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f070bfff700 1 -- 172.21.15.6:0/1720170973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0095f0 con 0x7f070c1002a0 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.838+0000 7f071099c700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f070bfff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06fc004730 tx=0x7f06fc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc01d070 con 0x7f070c1002a0 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06fc021410 con 0x7f070c1002a0 2023-11-17T00:07:43.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c10bc70 con 0x7f070c1002a0 2023-11-17T00:07:43.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc017650 con 0x7f070c1002a0 2023-11-17T00:07:43.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.839+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c10bae0 con 0x7f070c1002a0 2023-11-17T00:07:43.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.840+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c19e420 con 0x7f070c1002a0 2023-11-17T00:07:43.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.842+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f06fc021810 con 0x7f070c1002a0 2023-11-17T00:07:43.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.844+0000 7f0709ffb700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 0x7f06f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:43.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.844+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f06fc0ce2b0 con 0x7f070c1002a0 2023-11-17T00:07:43.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.845+0000 7f0703fff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 0x7f06f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:43.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.845+0000 7f0703fff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 0x7f06f80a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009c60 tx=0x7f06f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:43.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:43.849+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f06fc0975c0 con 0x7f070c1002a0 2023-11-17T00:07:44.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.057+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f070c107220 con 0x7f070c1002a0 2023-11-17T00:07:44.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.058+0000 7f0709ffb700 1 -- 172.21.15.6:0/1720170973 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06fc09abe0 con 0x7f070c1002a0 2023-11-17T00:07:44.060 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:07:44.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 msgr2=0x7f06f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 0x7f06f80a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009c60 tx=0x7f06f4009260 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 msgr2=0x7f070c196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06fc004730 tx=0x7f06fc004760 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 shutdown_connections 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f06f80a11e0 0x7f06f80a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f070c1002a0 0x7f070c196030 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f070c101e20 0x7f070c19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 --2- 172.21.15.6:0/1720170973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f070c101490 0x7f070c196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 >> 172.21.15.6:0/1720170973 conn(0x7f070c0fb880 msgr2=0x7f070c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.060+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 shutdown_connections 2023-11-17T00:07:44.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.061+0000 7f07123ff700 1 -- 172.21.15.6:0/1720170973 wait complete. 2023-11-17T00:07:44.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:07:44.143 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:43 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4028329832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.144 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:43 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/988968435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.288+0000 7f1a8ebff700 1 -- 172.21.15.6:0/818792765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881002a0 msgr2=0x7f1a881006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.288+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/818792765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881002a0 0x7f1a881006b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009990 tx=0x7f1a7c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 -- 172.21.15.6:0/818792765 shutdown_connections 2023-11-17T00:07:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/818792765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881002a0 0x7f1a881006b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/818792765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101e20 0x7f1a881086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/818792765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a88101490 0x7f1a881018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 -- 172.21.15.6:0/818792765 >> 172.21.15.6:0/818792765 conn(0x7f1a880fb880 msgr2=0x7f1a880fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 -- 172.21.15.6:0/818792765 shutdown_connections 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.289+0000 7f1a8ebff700 1 -- 172.21.15.6:0/818792765 wait complete. 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 Processor -- start 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 -- start start 2023-11-17T00:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 0x7f1a88198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101490 0x7f1a88198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8819fd90 con 0x7f1a88101e20 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8819fed0 con 0x7f1a881002a0 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.290+0000 7f1a8ebff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a881a0060 con 0x7f1a88101490 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36086/0 (socket says 172.21.15.6:36086) 2023-11-17T00:07:44.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 -- 172.21.15.6:0/1769139496 learned_addr learned my addr 172.21.15.6:0/1769139496 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8c99b700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 0x7f1a88198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a87fff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101490 0x7f1a88198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 -- 172.21.15.6:0/1769139496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101490 msgr2=0x7f1a88198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101490 0x7f1a88198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 -- 172.21.15.6:0/1769139496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 msgr2=0x7f1a88198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 0x7f1a88198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 -- 172.21.15.6:0/1769139496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a74009520 con 0x7f1a88101e20 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8c99b700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 0x7f1a88198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a8d19c700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ef30 tx=0x7f1a8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.291+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80018070 con 0x7f1a88101e20 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.292+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c0095f0 con 0x7f1a88101e20 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.292+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8000f040 con 0x7f1a88101e20 2023-11-17T00:07:44.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.292+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000cd00 con 0x7f1a88101e20 2023-11-17T00:07:44.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.292+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8810bc70 con 0x7f1a88101e20 2023-11-17T00:07:44.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.293+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8810bae0 con 0x7f1a88101e20 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.295+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1a80003a00 con 0x7f1a88101e20 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.296+0000 7f1a85ffb700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 0x7f1a780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.296+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1a800cd280 con 0x7f1a88101e20 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.296+0000 7f1a8c99b700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 0x7f1a780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.297+0000 7f1a8c99b700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 0x7f1a780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b2e0 tx=0x7f1a7c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:44.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.298+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a80096590 con 0x7f1a88101e20 2023-11-17T00:07:44.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:43 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4028329832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:43 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/988968435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:43 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4028329832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:43 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/988968435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.510+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a8810be00 con 0x7f1a88101e20 2023-11-17T00:07:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.510+0000 7f1a85ffb700 1 -- 172.21.15.6:0/1769139496 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a80094e60 con 0x7f1a88101e20 2023-11-17T00:07:44.512 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:07:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.512+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 msgr2=0x7f1a780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.512+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 0x7f1a780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b2e0 tx=0x7f1a7c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.512+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 msgr2=0x7f1a8819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.512+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000ef30 tx=0x7f1a8000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 shutdown_connections 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a780a11e0 0x7f1a780a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a88101e20 0x7f1a8819f7c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a881002a0 0x7f1a88198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 --2- 172.21.15.6:0/1769139496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a88101490 0x7f1a88198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 >> 172.21.15.6:0/1769139496 conn(0x7f1a880fb880 msgr2=0x7f1a88106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 shutdown_connections 2023-11-17T00:07:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.513+0000 7f1a8ebff700 1 -- 172.21.15.6:0/1769139496 wait complete. 2023-11-17T00:07:44.533 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.739+0000 7f42827ff700 1 -- 172.21.15.6:0/153311870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 msgr2=0x7f427c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.739+0000 7f42827ff700 1 --2- 172.21.15.6:0/153311870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c1018d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4270009990 tx=0x7f4270009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 -- 172.21.15.6:0/153311870 shutdown_connections 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 --2- 172.21.15.6:0/153311870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 0x7f427c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 --2- 172.21.15.6:0/153311870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 --2- 172.21.15.6:0/153311870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c1018d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 -- 172.21.15.6:0/153311870 >> 172.21.15.6:0/153311870 conn(0x7f427c0fb870 msgr2=0x7f427c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 -- 172.21.15.6:0/153311870 shutdown_connections 2023-11-17T00:07:44.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.740+0000 7f42827ff700 1 -- 172.21.15.6:0/153311870 wait complete. 2023-11-17T00:07:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 Processor -- start 2023-11-17T00:07:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 -- start start 2023-11-17T00:07:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 0x7f427c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c073bc0 con 0x7f427c101480 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c073d30 con 0x7f427c100290 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f42827ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c1a0070 con 0x7f427c101e10 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.741+0000 7f427b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f4280d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 0x7f427c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60182/0 (socket says 172.21.15.6:60182) 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 -- 172.21.15.6:0/894346205 learned_addr learned my addr 172.21.15.6:0/894346205 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 -- 172.21.15.6:0/894346205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 msgr2=0x7f427c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 0x7f427c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 -- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 msgr2=0x7f427c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 -- 172.21.15.6:0/894346205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42700095f0 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427b7fe700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f427bfff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d980 tx=0x7f426c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004020 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c00f690 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003a60 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.742+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c10bc50 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.743+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c10bac0 con 0x7f427c100290 2023-11-17T00:07:44.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.743+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c107240 con 0x7f427c100290 2023-11-17T00:07:44.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.745+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f426c00fb00 con 0x7f427c100290 2023-11-17T00:07:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.747+0000 7f42797fa700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 0x7f42640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.747+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f426c0cc980 con 0x7f427c100290 2023-11-17T00:07:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.747+0000 7f427b7fe700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 0x7f42640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.748+0000 7f427b7fe700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 0x7f42640a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f427000b5c0 tx=0x7f4270005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:44.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.749+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f426c095c00 con 0x7f427c100290 2023-11-17T00:07:44.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.958+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f427c061700 con 0x7f427c100290 2023-11-17T00:07:44.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.959+0000 7f42797fa700 1 -- 172.21.15.6:0/894346205 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f426c099220 con 0x7f427c100290 2023-11-17T00:07:44.961 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:07:44.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.961+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 msgr2=0x7f42640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.961+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 0x7f42640a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f427000b5c0 tx=0x7f4270005fd0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.961+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 msgr2=0x7f427c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:44.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.961+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d980 tx=0x7f426c00dc90 comp rx=0 tx=0).stop 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 shutdown_connections 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f42640a11e0 0x7f42640a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f427c101e10 0x7f427c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f427c100290 0x7f427c0748a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 --2- 172.21.15.6:0/894346205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f427c101480 0x7f427c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 >> 172.21.15.6:0/894346205 conn(0x7f427c0fb870 msgr2=0x7f427c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:44.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 shutdown_connections 2023-11-17T00:07:44.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:44.962+0000 7f42827ff700 1 -- 172.21.15.6:0/894346205 wait complete. 2023-11-17T00:07:44.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 -- 172.21.15.6:0/494904951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101490 msgr2=0x7fee1c1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 --2- 172.21.15.6:0/494904951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101490 0x7fee1c1018e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee04009990 tx=0x7fee04009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 -- 172.21.15.6:0/494904951 shutdown_connections 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 --2- 172.21.15.6:0/494904951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101e20 0x7fee1c1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 --2- 172.21.15.6:0/494904951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 0x7fee1c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 --2- 172.21.15.6:0/494904951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101490 0x7fee1c1018e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 -- 172.21.15.6:0/494904951 >> 172.21.15.6:0/494904951 conn(0x7fee1c0fb880 msgr2=0x7fee1c0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 -- 172.21.15.6:0/494904951 shutdown_connections 2023-11-17T00:07:45.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.194+0000 7fee20dff700 1 -- 172.21.15.6:0/494904951 wait complete. 2023-11-17T00:07:45.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.195+0000 7fee20dff700 1 Processor -- start 2023-11-17T00:07:45.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.195+0000 7fee20dff700 1 -- start start 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 0x7fee1c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 0x7fee1c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 0x7fee1c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19fd10 con 0x7fee1c101490 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19fe50 con 0x7fee1c1002a0 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee20dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c19ffe0 con 0x7fee1c101e20 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee1ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 0x7fee1c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 0x7fee1c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee1ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 0x7fee1c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44746/0 (socket says 172.21.15.6:44746) 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee1a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 0x7fee1c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee1ad9d700 1 -- 172.21.15.6:0/1569745788 learned_addr learned my addr 172.21.15.6:0/1569745788 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 -- 172.21.15.6:0/1569745788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 msgr2=0x7fee1c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 0x7fee1c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 -- 172.21.15.6:0/1569745788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 msgr2=0x7fee1c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 0x7fee1c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.196+0000 7fee13fff700 1 -- 172.21.15.6:0/1569745788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14009c40 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee13fff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 0x7fee1c1986d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee040098e0 tx=0x7fee04009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0401d070 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040095f0 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee04021410 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04005720 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.197+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c10bc00 con 0x7fee1c101490 2023-11-17T00:07:45.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.198+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c067a50 con 0x7fee1c101490 2023-11-17T00:07:45.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.200+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fee04021580 con 0x7fee1c101490 2023-11-17T00:07:45.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.201+0000 7fee137fe700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 0x7fee080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.201+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee040ce0c0 con 0x7fee1c101490 2023-11-17T00:07:45.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.202+0000 7fee1a59c700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 0x7fee080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.202+0000 7fee1a59c700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 0x7fee080a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee1c1012f0 tx=0x7fee0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:45.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.205+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee040973d0 con 0x7fee1c101490 2023-11-17T00:07:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:44 smithi006 ceph-mon[97840]: pgmap v341: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:44 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1720170973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:44 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1769139496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.350 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:44 smithi087 ceph-mon[131182]: pgmap v341: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:44 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1720170973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:44 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1769139496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:44 smithi186 ceph-mon[131487]: pgmap v341: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:44 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1720170973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:44 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1769139496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:07:45.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.416+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fee1c1a0a50 con 0x7fee1c101490 2023-11-17T00:07:45.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.417+0000 7fee137fe700 1 -- 172.21.15.6:0/1569745788 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fee0409a9f0 con 0x7fee1c101490 2023-11-17T00:07:45.419 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 msgr2=0x7fee080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 0x7fee080a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee1c1012f0 tx=0x7fee0c009260 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 msgr2=0x7fee1c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 0x7fee1c1986d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee040098e0 tx=0x7fee04009580 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 shutdown_connections 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fee080a11e0 0x7fee080a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee1c101e20 0x7fee1c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee1c1002a0 0x7fee1c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 --2- 172.21.15.6:0/1569745788 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee1c101490 0x7fee1c1986d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 >> 172.21.15.6:0/1569745788 conn(0x7fee1c0fb880 msgr2=0x7fee1c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 shutdown_connections 2023-11-17T00:07:45.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.419+0000 7fee20dff700 1 -- 172.21.15.6:0/1569745788 wait complete. 2023-11-17T00:07:45.440 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:07:45.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 -- 172.21.15.6:0/1332114274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c1014c0 msgr2=0x7f656c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 --2- 172.21.15.6:0/1332114274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c1014c0 0x7f656c101910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6560009990 tx=0x7f6560009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 -- 172.21.15.6:0/1332114274 shutdown_connections 2023-11-17T00:07:45.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 --2- 172.21.15.6:0/1332114274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 --2- 172.21.15.6:0/1332114274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c101e50 0x7f656c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 --2- 172.21.15.6:0/1332114274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c1014c0 0x7f656c101910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 -- 172.21.15.6:0/1332114274 >> 172.21.15.6:0/1332114274 conn(0x7f656c0fb8b0 msgr2=0x7f656c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:45.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 -- 172.21.15.6:0/1332114274 shutdown_connections 2023-11-17T00:07:45.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.643+0000 7f6571dff700 1 -- 172.21.15.6:0/1332114274 wait complete. 2023-11-17T00:07:45.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.644+0000 7f6571dff700 1 Processor -- start 2023-11-17T00:07:45.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.644+0000 7f6571dff700 1 -- start start 2023-11-17T00:07:45.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c1014c0 0x7f656c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 0x7f656c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c073bc0 con 0x7f656c1002d0 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c073d30 con 0x7f656c101e50 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f6571dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1a0090 con 0x7f656c1014c0 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36142/0 (socket says 172.21.15.6:36142) 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 -- 172.21.15.6:0/4068867831 learned_addr learned my addr 172.21.15.6:0/4068867831 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656bfff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 0x7f656c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656affd700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c1014c0 0x7f656c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 -- 172.21.15.6:0/4068867831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c1014c0 msgr2=0x7f656c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c1014c0 0x7f656c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 -- 172.21.15.6:0/4068867831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 msgr2=0x7f656c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 0x7f656c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656b7fe700 1 -- 172.21.15.6:0/4068867831 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600095f0 con 0x7f656c1002d0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.645+0000 7f656bfff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 0x7f656c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f656b7fe700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e970 tx=0x7f655c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c009790 con 0x7f656c1002d0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c10bc80 con 0x7f656c1002d0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655c0044e0 con 0x7f656c1002d0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c003e30 con 0x7f656c1002d0 2023-11-17T00:07:45.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.646+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c10baf0 con 0x7f656c1002d0 2023-11-17T00:07:45.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.647+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c107840 con 0x7f656c1002d0 2023-11-17T00:07:45.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.649+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f655c004000 con 0x7f656c1002d0 2023-11-17T00:07:45.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.650+0000 7f6568ff9700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 0x7f65540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:45.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.651+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f655c014070 con 0x7f656c1002d0 2023-11-17T00:07:45.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.651+0000 7f656affd700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 0x7f65540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:45.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.651+0000 7f656affd700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 0x7f65540a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f656000b2e0 tx=0x7f6560005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:45.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.654+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f655c096360 con 0x7f656c1002d0 2023-11-17T00:07:45.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.915+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f656c02cfe0 con 0x7f656c1002d0 2023-11-17T00:07:45.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.915+0000 7f6568ff9700 1 -- 172.21.15.6:0/4068867831 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f655c099980 con 0x7f656c1002d0 2023-11-17T00:07:45.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:45.918 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:07:45.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 msgr2=0x7f65540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 0x7f65540a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f656000b2e0 tx=0x7f6560005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 msgr2=0x7f656c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e970 tx=0x7f655c00ed30 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 shutdown_connections 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f65540a11e0 0x7f65540a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c1002d0 0x7f656c0748a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f656c101e50 0x7f656c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 --2- 172.21.15.6:0/4068867831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f656c1014c0 0x7f656c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 >> 172.21.15.6:0/4068867831 conn(0x7f656c0fb8b0 msgr2=0x7f656c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 shutdown_connections 2023-11-17T00:07:45.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:45.918+0000 7f6571dff700 1 -- 172.21.15.6:0/4068867831 wait complete. 2023-11-17T00:07:45.921 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:07:45.944 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 -- 172.21.15.6:0/4021024562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 msgr2=0x7facd0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 --2- 172.21.15.6:0/4021024562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0108710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7facc800b0a0 tx=0x7facc800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 -- 172.21.15.6:0/4021024562 shutdown_connections 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 --2- 172.21.15.6:0/4021024562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0108710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 --2- 172.21.15.6:0/4021024562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 --2- 172.21.15.6:0/4021024562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 0x7facd0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 -- 172.21.15.6:0/4021024562 >> 172.21.15.6:0/4021024562 conn(0x7facd00fb8b0 msgr2=0x7facd00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 -- 172.21.15.6:0/4021024562 shutdown_connections 2023-11-17T00:07:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.147+0000 7facd7bff700 1 -- 172.21.15.6:0/4021024562 wait complete. 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.148+0000 7facd7bff700 1 Processor -- start 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.148+0000 7facd7bff700 1 -- start start 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.148+0000 7facd7bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd7bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 0x7facd0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd7bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd7bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0073bc0 con 0x7facd0101e50 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd7bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0073d30 con 0x7facd01014c0 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd7bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01a01b0 con 0x7facd01002d0 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd619c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44784/0 (socket says 172.21.15.6:44784) 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 -- 172.21.15.6:0/837674384 learned_addr learned my addr 172.21.15.6:0/837674384 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:46.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd519a700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 0x7facd0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 -- 172.21.15.6:0/837674384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 msgr2=0x7facd0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 0x7facd0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 -- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 msgr2=0x7facd0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd599b700 1 -- 172.21.15.6:0/837674384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009d00 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.149+0000 7facd619c700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facd599b700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e970 tx=0x7faccc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc009790 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faccc00c7e0 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd010bca0 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc003c70 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.150+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd010bb10 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.151+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd01077d0 con 0x7facd01002d0 2023-11-17T00:07:46.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.153+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7faccc00cc50 con 0x7facd01002d0 2023-11-17T00:07:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.154+0000 7facc6ffd700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 0x7facbc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.154+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faccc014070 con 0x7facd01002d0 2023-11-17T00:07:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.155+0000 7facd519a700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 0x7facbc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.155+0000 7facd519a700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 0x7facbc0a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7facc0009c60 tx=0x7facc0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.158+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faccc096220 con 0x7facd01002d0 2023-11-17T00:07:46.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:45 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/894346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:45 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1569745788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:45 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/894346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:45 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1569745788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:45 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/894346205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:45 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1569745788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:07:46.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.366+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facd002cfe0 con 0x7facd01002d0 2023-11-17T00:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.368+0000 7facc6ffd700 1 -- 172.21.15.6:0/837674384 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7faccc003dd0 con 0x7facd01002d0 2023-11-17T00:07:46.370 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:46.370 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":90,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:07:02.818255+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.370+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 msgr2=0x7facbc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.370+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 0x7facbc0a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7facc0009c60 tx=0x7facc0009260 comp rx=0 tx=0).stop 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.370+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 msgr2=0x7facd00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.370+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e970 tx=0x7faccc00ec80 comp rx=0 tx=0).stop 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 shutdown_connections 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7facbc0a11e0 0x7facbc0a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd0101e50 0x7facd0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd01002d0 0x7facd00748a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 --2- 172.21.15.6:0/837674384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd01014c0 0x7facd0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 >> 172.21.15.6:0/837674384 conn(0x7facd00fb8b0 msgr2=0x7facd0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 shutdown_connections 2023-11-17T00:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.371+0000 7facd7bff700 1 -- 172.21.15.6:0/837674384 wait complete. 2023-11-17T00:07:46.392 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 -- 172.21.15.6:0/1482454114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b01002d0 msgr2=0x7ff3b01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/1482454114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b01002d0 0x7ff3b01006e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009990 tx=0x7ff3ac009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 -- 172.21.15.6:0/1482454114 shutdown_connections 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/1482454114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b01002d0 0x7ff3b01006e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/1482454114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b0101e50 0x7ff3b0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/1482454114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01014c0 0x7ff3b0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 -- 172.21.15.6:0/1482454114 >> 172.21.15.6:0/1482454114 conn(0x7ff3b00fb8b0 msgr2=0x7ff3b00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 -- 172.21.15.6:0/1482454114 shutdown_connections 2023-11-17T00:07:46.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.594+0000 7ff3b7fff700 1 -- 172.21.15.6:0/1482454114 wait complete. 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.595+0000 7ff3b7fff700 1 Processor -- start 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.595+0000 7ff3b7fff700 1 -- start start 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01002d0 0x7ff3b00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b01014c0 0x7ff3b0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0073bc0 con 0x7ff3b0101e50 2023-11-17T00:07:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0073d30 con 0x7ff3b01002d0 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01a00c0 con 0x7ff3b01014c0 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b01014c0 0x7ff3b0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36172/0 (socket says 172.21.15.6:36172) 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 -- 172.21.15.6:0/2489116654 learned_addr learned my addr 172.21.15.6:0/2489116654 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b5d9b700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01002d0 0x7ff3b00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 -- 172.21.15.6:0/2489116654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b01014c0 msgr2=0x7ff3b0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b01014c0 0x7ff3b0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 -- 172.21.15.6:0/2489116654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01002d0 msgr2=0x7ff3b00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01002d0 0x7ff3b00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.596+0000 7ff3b659c700 1 -- 172.21.15.6:0/2489116654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0095f0 con 0x7ff3b0101e50 2023-11-17T00:07:46.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3b659c700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800ef30 tx=0x7ff3a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8018070 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a800f040 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800cd00 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b010bca0 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.597+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b010bb10 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.598+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0107800 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.600+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff3a8003a00 con 0x7ff3b0101e50 2023-11-17T00:07:46.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.601+0000 7ff3a6ffd700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 0x7ff39c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.601+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3a80cc930 con 0x7ff3b0101e50 2023-11-17T00:07:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.602+0000 7ff3b5d9b700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 0x7ff39c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.602+0000 7ff3b5d9b700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 0x7ff39c0a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00b2e0 tx=0x7ff3ac005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.605+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff3a8095c40 con 0x7ff3b0101e50 2023-11-17T00:07:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.862+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3b002cfe0 con 0x7ff3b0101e50 2023-11-17T00:07:46.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.863+0000 7ff3a6ffd700 1 -- 172.21.15.6:0/2489116654 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7ff3a8099260 con 0x7ff3b0101e50 2023-11-17T00:07:46.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:07:46.865 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:07:46.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 msgr2=0x7ff39c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 0x7ff39c0a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00b2e0 tx=0x7ff3ac005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 msgr2=0x7ff3b0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:46.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800ef30 tx=0x7ff3a800c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 shutdown_connections 2023-11-17T00:07:46.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ff39c0a11e0 0x7ff39c0a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3b0101e50 0x7ff3b0073420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3b01002d0 0x7ff3b00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 --2- 172.21.15.6:0/2489116654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b01014c0 0x7ff3b0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:46.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 >> 172.21.15.6:0/2489116654 conn(0x7ff3b00fb8b0 msgr2=0x7ff3b0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:46.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 shutdown_connections 2023-11-17T00:07:46.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:46.866+0000 7ff3b7fff700 1 -- 172.21.15.6:0/2489116654 wait complete. 2023-11-17T00:07:46.873 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:07:46.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-17T00:06:12.598183+0000', 'modified': '2023-11-17T00:06:37.833900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6837/850260336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 850260336}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 850260336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.87:6839/804288298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 804288298}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 804288298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-17T00:07:46.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-17T00:07:46.887 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-17T00:07:47.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/336393685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64102470 msgr2=0x7f0b64102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/336393685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64102470 0x7f0b64102880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009990 tx=0x7f0b60009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/336393685 shutdown_connections 2023-11-17T00:07:47.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/336393685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6410a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/336393685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64102470 0x7f0b64102880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/336393685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64103660 0x7f0b64103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/336393685 >> 172.21.15.6:0/336393685 conn(0x7f0b640fda90 msgr2=0x7f0b640ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/336393685 shutdown_connections 2023-11-17T00:07:47.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.091+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/336393685 wait complete. 2023-11-17T00:07:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.092+0000 7f0b6c3ff700 1 Processor -- start 2023-11-17T00:07:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.092+0000 7f0b6c3ff700 1 -- start start 2023-11-17T00:07:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64102470 0x7f0b641981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 0x7f0b64198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419fd40 con 0x7f0b64103ff0 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419fe80 con 0x7f0b64103660 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b641a0010 con 0x7f0b64102470 2023-11-17T00:07:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 0x7f0b64198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 0x7f0b64198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60252/0 (socket says 172.21.15.6:60252) 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6999a700 1 -- 172.21.15.6:0/1966945413 learned_addr learned my addr 172.21.15.6:0/1966945413 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64102470 0x7f0b641981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.093+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1966945413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64102470 msgr2=0x7f0b641981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64102470 0x7f0b641981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1966945413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 msgr2=0x7f0b64198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 0x7f0b64198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1966945413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600095f0 con 0x7f0b64103ff0 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6419f770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00efc0 tx=0x7f0b5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b5b7fe700 1 -- 172.21.15.6:0/1966945413 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c018070 con 0x7f0b64103ff0 2023-11-17T00:07:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b5b7fe700 1 -- 172.21.15.6:0/1966945413 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b5c00f040 con 0x7f0b64103ff0 2023-11-17T00:07:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6410de20 con 0x7f0b64103ff0 2023-11-17T00:07:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b5b7fe700 1 -- 172.21.15.6:0/1966945413 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00cd40 con 0x7f0b64103ff0 2023-11-17T00:07:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.094+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6410dc90 con 0x7f0b64103ff0 2023-11-17T00:07:47.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.097+0000 7f0b5b7fe700 1 -- 172.21.15.6:0/1966945413 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0b5c003a00 con 0x7f0b64103ff0 2023-11-17T00:07:47.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.098+0000 7f0b7fc1fb80 1 Processor -- start 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 -- start start 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b5b7fe700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 0x7f0b500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b5b7fe700 1 -- 172.21.15.6:0/1966945413 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b5c0cc8e0 con 0x7f0b64103ff0 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43ba0c0 0x5617f43ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f42be010 con 0x5617f43b8bb0 2023-11-17T00:07:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f4310850 con 0x5617f43b9500 2023-11-17T00:07:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f4309ce0 con 0x5617f43ba0c0 2023-11-17T00:07:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b6999a700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 0x7f0b500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b6a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b6a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36188/0 (socket says 172.21.15.6:36188) 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.099+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2355787356 learned_addr learned my addr 172.21.15.6:0/2355787356 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43ba0c0 0x5617f43ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2355787356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43ba0c0 msgr2=0x5617f43ba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43ba0c0 0x5617f43ba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2355787356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 msgr2=0x5617f43b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43b9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2355787356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f418b5e0 con 0x5617f43b8bb0 2023-11-17T00:07:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6999a700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 0x7f0b500a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009880 tx=0x7f0b54009c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b641034c0 tx=0x7f0b60005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2244b0e70e99625 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b58bfe700 1 -- 172.21.15.6:0/2355787356 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001d070 con 0x5617f43b8bb0 2023-11-17T00:07:47.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b58bfe700 1 -- 172.21.15.6:0/2355787356 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b6001e040 con 0x5617f43b8bb0 2023-11-17T00:07:47.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.100+0000 7f0b58bfe700 1 -- 172.21.15.6:0/2355787356 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600047d0 con 0x5617f43b8bb0 2023-11-17T00:07:47.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2355787356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 msgr2=0x5617f43b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b641034c0 tx=0x7f0b60005ea0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2355787356 shutdown_connections 2023-11-17T00:07:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b8bb0 0x5617f43b8fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43ba0c0 0x5617f43ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2355787356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43b9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2355787356 >> 172.21.15.6:0/2355787356 conn(0x5617f43ae310 msgr2=0x5617f43b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2355787356 shutdown_connections 2023-11-17T00:07:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.101+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2355787356 wait complete. 2023-11-17T00:07:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.102+0000 7f0b7fc1fb80 1 -- start start 2023-11-17T00:07:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.102+0000 7f0b7fc1fb80 1 Processor -- start 2023-11-17T00:07:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f42ab890 con 0x5617f43b9500 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f42ab530 con 0x5617f43ba0c0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f418d8d0 con 0x5617f43b8bb0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b7fc1fb80 10 client.0 _ll_register_callbacks cb 0x5617f42c9730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b6a99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b6a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b6a99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60266/0 (socket says 172.21.15.6:60266) 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.103+0000 7f0b6a99c700 1 -- 172.21.15.6:0/4269920552 learned_addr learned my addr 172.21.15.6:0/4269920552 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 -- 172.21.15.6:0/4269920552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 msgr2=0x5617f43c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 -- 172.21.15.6:0/4269920552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 msgr2=0x5617f43c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 -- 172.21.15.6:0/4269920552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6410b1e0 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0c9f50 tx=0x7f0b5c099290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd21e383a33b2fd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c098fa0 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617f43abad0 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.104+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b5c0998b0 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5c09d8e0 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 10 client.25139 fetch_fsmap learned FSMap version 20 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5617f42d0140 con 0x5617f43ba0c0 2023-11-17T00:07:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617f43abad0 con 0x5617f43ba0c0 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b5c0b9100 con 0x5617f43ba0c0 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 10 client.25139 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 10 client.25139 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 20 client.25139 populate_metadata read hostname 'smithi006' 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f43cd640 0x5617f43ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 4 client.25139 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.105+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f42cc050 con 0x5617f43cd640 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.106+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f43cd640 0x5617f43ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.106+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f43cd640 0x5617f43ccad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127466c3e63c553d server_cookie=3a299375967a7f6c in_seq=0 out_seq=0 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.106+0000 7f0b4dffb700 10 client.25139 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b4dffb700 1 -- 172.21.15.6:0/4269920552 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617f42cc050 con 0x5617f43cd640 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b4dffb700 10 client.25139 handle_command_reply: tid=0 2023-11-17T00:07:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 client.25139 shutdown 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 msgr2=0x5617f43c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0c9f50 tx=0x7f0b5c099290 comp rx=0 tx=0).stop 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 shutdown_connections 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9a30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f43cd640 0x5617f43ccad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/4269920552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b4dffb700 0 client.25139 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 >> 172.21.15.6:0/4269920552 conn(0x5617f43ae310 msgr2=0x5617f43b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 shutdown_connections 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.107+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/4269920552 wait complete. 2023-11-17T00:07:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.108+0000 7f0b7fc1fb80 20 client.25139 trim_cache size 0 max 16384 2023-11-17T00:07:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.126+0000 7f0b7fc1fb80 1 Processor -- start 2023-11-17T00:07:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.126+0000 7f0b7fc1fb80 1 -- start start 2023-11-17T00:07:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.126+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f43c2080 con 0x5617f43b9500 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f42ab530 con 0x5617f43ba0c0 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f43cd360 con 0x5617f43b8bb0 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6999a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36210/0 (socket says 172.21.15.6:36210) 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6999a700 1 -- 172.21.15.6:0/2569135094 learned_addr learned my addr 172.21.15.6:0/2569135094 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44864/0 (socket says 172.21.15.6:44864) 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2569135094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 msgr2=0x5617f43c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2569135094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 msgr2=0x5617f43bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6a19b700 1 -- 172.21.15.6:0/2569135094 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f42cfe90 con 0x5617f43b8bb0 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.127+0000 7f0b6999a700 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098e0 tx=0x7f0b6001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24689a328d2512be server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b593ff700 1 -- 172.21.15.6:0/2569135094 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60032070 con 0x5617f43b8bb0 2023-11-17T00:07:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b593ff700 1 -- 172.21.15.6:0/2569135094 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b60037070 con 0x5617f43b8bb0 2023-11-17T00:07:47.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b593ff700 1 -- 172.21.15.6:0/2569135094 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001d040 con 0x5617f43b8bb0 2023-11-17T00:07:47.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2569135094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 msgr2=0x5617f43bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.128+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098e0 tx=0x7f0b6001a040 comp rx=0 tx=0).stop 2023-11-17T00:07:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2569135094 shutdown_connections 2023-11-17T00:07:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43ba0c0 0x5617f43c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43bcf70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/2569135094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43b9500 0x5617f43bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2569135094 >> 172.21.15.6:0/2569135094 conn(0x5617f43ae310 msgr2=0x5617f43c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2569135094 shutdown_connections 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/2569135094 wait complete. 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 -- start start 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.129+0000 7f0b7fc1fb80 1 Processor -- start 2023-11-17T00:07:47.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f43c9b50 con 0x5617f43ba0c0 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f43c9cc0 con 0x5617f43b9500 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f43c9e80 con 0x5617f43b8bb0 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b7fc1fb80 10 client.0 _ll_register_callbacks cb 0x7f0b5c0996a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36220/0 (socket says 172.21.15.6:36220) 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1521694201 learned_addr learned my addr 172.21.15.6:0/1521694201 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:47.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6999a700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1521694201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 msgr2=0x5617f43c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 msgr2=0x5617f43c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a99c700 1 -- 172.21.15.6:0/1521694201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5401b040 con 0x5617f43ba0c0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.130+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b6a99c700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c095db0 tx=0x7f0b5c0a1a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf9308143db5dd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c095a20 con 0x5617f43ba0c0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b6001e040 con 0x5617f43ba0c0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b5c0998b0 con 0x5617f43ba0c0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5c09d8e0 con 0x5617f43ba0c0 2023-11-17T00:07:47.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 10 client.15093 fetch_fsmap learned FSMap version 20 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5617f418b5e0 con 0x5617f43ba0c0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b6001e040 con 0x5617f43ba0c0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b5c084170 con 0x5617f43ba0c0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 10 client.15093 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.131+0000 7f0b7fc1fb80 20 client.15093 populate_metadata read hostname 'smithi006' 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f44261d0 0x5617f4422b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b7fc1fb80 4 client.15093 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617f42d8610 con 0x5617f44261d0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f44261d0 0x5617f4422b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b4dffb700 10 client.15093 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b6a19b700 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f44261d0 0x5617f4422b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9882009a9e2f5a server_cookie=81f9c202e07bad16 in_seq=0 out_seq=0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b4dffb700 1 -- 172.21.15.6:0/1521694201 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5617f42d8610 con 0x5617f44261d0 2023-11-17T00:07:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b4dffb700 10 client.15093 handle_command_reply: tid=0 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.132+0000 7f0b7fc1fb80 1 client.15093 shutdown 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 msgr2=0x5617f4421500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c095db0 tx=0x7f0b5c0a1a10 comp rx=0 tx=0).stop 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 shutdown_connections 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5617f44261d0 0x5617f4422b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5617f43ba0c0 0x5617f4421500 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5617f43b8bb0 0x5617f43c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 --2- 172.21.15.6:0/1521694201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5617f43b9500 0x5617f43c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b4dffb700 0 client.15093 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 >> 172.21.15.6:0/1521694201 conn(0x5617f43ae310 msgr2=0x5617f43bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 shutdown_connections 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 1 -- 172.21.15.6:0/1521694201 wait complete. 2023-11-17T00:07:47.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.133+0000 7f0b7fc1fb80 20 client.15093 trim_cache size 0 max 16384 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dbc2f8cc-84db-11ee-8dbe-212e2dc638e7", 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24578, 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 94.532231048, 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 20, 2023-11-17T00:07:47.151 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 90, 2023-11-17T00:07:47.152 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 88, 2023-11-17T00:07:47.152 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 106.69268266500001 2023-11-17T00:07:47.152 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:07:47.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 msgr2=0x7f0b500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 0x7f0b500a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009880 tx=0x7f0b54009c40 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 msgr2=0x7f0b6419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6419f770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00efc0 tx=0x7f0b5c00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 shutdown_connections 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f0b500a1260 0x7f0b500a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b64103ff0 0x7f0b6419f770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b64102470 0x7f0b641981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 --2- 172.21.15.6:0/1966945413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b64103660 0x7f0b64198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 >> 172.21.15.6:0/1966945413 conn(0x7f0b640fda90 msgr2=0x7f0b64106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 shutdown_connections 2023-11-17T00:07:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:47.134+0000 7f0b6c3ff700 1 -- 172.21.15.6:0/1966945413 wait complete. 2023-11-17T00:07:47.153 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:46 smithi006 ceph-mon[97840]: pgmap v342: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:47.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:46 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4068867831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:46 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/837674384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:46 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2489116654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.155 INFO:tasks.cephadm:Setup complete, yielding 2023-11-17T00:07:47.155 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-17T00:07:47.244 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:07:47.245 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch status' 2023-11-17T00:07:47.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:46 smithi087 ceph-mon[131182]: pgmap v342: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:47.508 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:46 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4068867831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.509 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:46 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/837674384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.509 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:46 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2489116654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:46 smithi186 ceph-mon[131487]: pgmap v342: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:47.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:46 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4068867831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.509 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:46 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/837674384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:07:47.510 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:46 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2489116654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:07:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:49.227 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:48 smithi006 ceph-mon[97840]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:48 smithi087 ceph-mon[131182]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:48 smithi186 ceph-mon[131487]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:50.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.019+0000 7fe0283c5700 1 -- 172.21.15.6:0/414952245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201014f0 msgr2=0x7fe020101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:50.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.019+0000 7fe0283c5700 1 --2- 172.21.15.6:0/414952245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201014f0 0x7fe020101940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe010009990 tx=0x7fe010009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 -- 172.21.15.6:0/414952245 shutdown_connections 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 --2- 172.21.15.6:0/414952245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 0x7fe020108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 --2- 172.21.15.6:0/414952245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201002f0 0x7fe020100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 --2- 172.21.15.6:0/414952245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201014f0 0x7fe020101940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 -- 172.21.15.6:0/414952245 >> 172.21.15.6:0/414952245 conn(0x7fe0200fb900 msgr2=0x7fe0200fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 -- 172.21.15.6:0/414952245 shutdown_connections 2023-11-17T00:07:50.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.020+0000 7fe0283c5700 1 -- 172.21.15.6:0/414952245 wait complete. 2023-11-17T00:07:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 Processor -- start 2023-11-17T00:07:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 -- start start 2023-11-17T00:07:50.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 0x7fe020198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 0x7fe020198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 0x7fe02019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02019fdc0 con 0x7fe0201002f0 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02019ff00 con 0x7fe0201014f0 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe0283c5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201a0090 con 0x7fe020101e80 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe026161700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 0x7fe020198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe025960700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 0x7fe020198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe026161700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 0x7fe020198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36036/0 (socket says 172.21.15.6:36036) 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.021+0000 7fe026161700 1 -- 172.21.15.6:0/2052797189 learned_addr learned my addr 172.21.15.6:0/2052797189 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe026962700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 0x7fe02019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 -- 172.21.15.6:0/2052797189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 msgr2=0x7fe02019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 0x7fe02019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 -- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 msgr2=0x7fe020198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 0x7fe020198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 -- 172.21.15.6:0/2052797189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100095f0 con 0x7fe0201014f0 2023-11-17T00:07:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.022+0000 7fe025960700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 0x7fe020198770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0100098e0 tx=0x7fe010009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.023+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01001d070 con 0x7fe0201014f0 2023-11-17T00:07:50.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.023+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02010bc90 con 0x7fe0201014f0 2023-11-17T00:07:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.023+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe010021410 con 0x7fe0201014f0 2023-11-17T00:07:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.023+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010005720 con 0x7fe0201014f0 2023-11-17T00:07:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.023+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02010bb00 con 0x7fe0201014f0 2023-11-17T00:07:50.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.024+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0201929f0 con 0x7fe0201014f0 2023-11-17T00:07:50.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.026+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe010021580 con 0x7fe0201014f0 2023-11-17T00:07:50.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.027+0000 7fe0177fe700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 0x7fe00c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:50.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.027+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe0100ce200 con 0x7fe0201014f0 2023-11-17T00:07:50.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.027+0000 7fe026161700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 0x7fe00c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:50.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.028+0000 7fe026161700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 0x7fe00c0a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005950 tx=0x7fe01c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:50.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.030+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe010097510 con 0x7fe0201014f0 2023-11-17T00:07:50.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.251+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe020061bf0 con 0x7fe00c0a11e0 2023-11-17T00:07:50.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.252+0000 7fe0177fe700 1 -- 172.21.15.6:0/2052797189 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe020061bf0 con 0x7fe00c0a11e0 2023-11-17T00:07:50.254 INFO:teuthology.orchestra.run.smithi006.stdout:Backend: cephadm 2023-11-17T00:07:50.254 INFO:teuthology.orchestra.run.smithi006.stdout:Available: Yes 2023-11-17T00:07:50.254 INFO:teuthology.orchestra.run.smithi006.stdout:Paused: No 2023-11-17T00:07:50.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 msgr2=0x7fe00c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 0x7fe00c0a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005950 tx=0x7fe01c009260 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 msgr2=0x7fe020198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 0x7fe020198770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0100098e0 tx=0x7fe010009580 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 shutdown_connections 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe00c0a11e0 0x7fe00c0a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0201002f0 0x7fe020198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020101e80 0x7fe02019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 --2- 172.21.15.6:0/2052797189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0201014f0 0x7fe020198770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 >> 172.21.15.6:0/2052797189 conn(0x7fe0200fb900 msgr2=0x7fe020106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 shutdown_connections 2023-11-17T00:07:50.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:50.255+0000 7fe0283c5700 1 -- 172.21.15.6:0/2052797189 wait complete. 2023-11-17T00:07:50.915 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-11-17T00:07:51.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:50 smithi006 ceph-mon[97840]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:51.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:50 smithi087 ceph-mon[131182]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:51.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:50 smithi186 ceph-mon[131487]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:51.777 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:52.184 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:51 smithi006 ceph-mon[97840]: from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:52.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:51 smithi087 ceph-mon[131182]: from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:52.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:51 smithi186 ceph-mon[131487]: from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:52 smithi006 ceph-mon[97840]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:52 smithi087 ceph-mon[131182]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:53.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:52 smithi186 ceph-mon[131487]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:53.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 -- 172.21.15.6:0/1588448880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 msgr2=0x7feb44108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:53.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 --2- 172.21.15.6:0/1588448880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb44108740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b0a0 tx=0x7feb3c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 -- 172.21.15.6:0/1588448880 shutdown_connections 2023-11-17T00:07:53.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 --2- 172.21.15.6:0/1588448880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb44108740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 --2- 172.21.15.6:0/1588448880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441002f0 0x7feb44100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 --2- 172.21.15.6:0/1588448880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441014f0 0x7feb44101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 -- 172.21.15.6:0/1588448880 >> 172.21.15.6:0/1588448880 conn(0x7feb440fb900 msgr2=0x7feb440fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:53.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 -- 172.21.15.6:0/1588448880 shutdown_connections 2023-11-17T00:07:53.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.718+0000 7feb4b1df700 1 -- 172.21.15.6:0/1588448880 wait complete. 2023-11-17T00:07:53.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.719+0000 7feb4b1df700 1 Processor -- start 2023-11-17T00:07:53.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 -- start start 2023-11-17T00:07:53.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441002f0 0x7feb44198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:53.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 0x7feb44198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:53.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:53.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4419fed0 con 0x7feb44101e80 2023-11-17T00:07:53.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441a0010 con 0x7feb441014f0 2023-11-17T00:07:53.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4b1df700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441a01a0 con 0x7feb441002f0 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4977c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4977c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36054/0 (socket says 172.21.15.6:36054) 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb4977c700 1 -- 172.21.15.6:0/540154478 learned_addr learned my addr 172.21.15.6:0/540154478 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb43fff700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 0x7feb44198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.720+0000 7feb48f7b700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441002f0 0x7feb44198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 -- 172.21.15.6:0/540154478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441002f0 msgr2=0x7feb44198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:53.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441002f0 0x7feb44198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 -- 172.21.15.6:0/540154478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 msgr2=0x7feb44198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:53.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 0x7feb44198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 -- 172.21.15.6:0/540154478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c009d00 con 0x7feb44101e80 2023-11-17T00:07:53.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb43fff700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 0x7feb44198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4977c700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b070 tx=0x7feb3c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00e040 con 0x7feb44101e80 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3c0094a0 con 0x7feb44101e80 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4410bcb0 con 0x7feb44101e80 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c01b6c0 con 0x7feb44101e80 2023-11-17T00:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.721+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4410bb20 con 0x7feb44101e80 2023-11-17T00:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.723+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb44066ea0 con 0x7feb44101e80 2023-11-17T00:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.725+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7feb3c01d470 con 0x7feb44101e80 2023-11-17T00:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.726+0000 7feb41ffb700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 0x7feb340a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.726+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb3c0cdcf0 con 0x7feb44101e80 2023-11-17T00:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.726+0000 7feb48f7b700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 0x7feb340a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:53.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.727+0000 7feb48f7b700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 0x7feb340a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb44101350 tx=0x7feb38009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:53.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.730+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feb3c097000 con 0x7feb44101e80 2023-11-17T00:07:53.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.952+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb441a0d30 con 0x7feb340a1260 2023-11-17T00:07:53.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.966+0000 7feb41ffb700 1 -- 172.21.15.6:0/540154478 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7feb441a0d30 con 0x7feb340a1260 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.a smithi006 running (116s) 26s ago 115s 13.7M - 18.0.0-3906-g1c956514 ace6904b066b 11f75eebf121 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.b smithi087 running (113s) 41s ago 113s 30.3M - 18.0.0-3906-g1c956514 ace6904b066b 725d4b84a798 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.c smithi186 running (2m) 41s ago 2m 17.6M - 18.0.0-3906-g1c956514 ace6904b066b 4fe53db931cb 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.d smithi006 running (2m) 26s ago 2m 13.8M - 18.0.0-3906-g1c956514 ace6904b066b 52b1941b78f8 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.e smithi087 running (2m) 41s ago 2m 16.6M - 18.0.0-3906-g1c956514 ace6904b066b eb76dc3eda17 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.f smithi186 running (2m) 41s ago 2m 15.1M - 18.0.0-3906-g1c956514 ace6904b066b 346bf7dfc39e 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.g smithi006 running (119s) 26s ago 118s 13.8M - 18.0.0-3906-g1c956514 ace6904b066b 4212766931ac 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.h smithi087 running (2m) 41s ago 2m 21.1M - 18.0.0-3906-g1c956514 ace6904b066b 1fba86f56a14 2023-11-17T00:07:53.969 INFO:teuthology.orchestra.run.smithi006.stdout:mds.i smithi186 running (2m) 41s ago 2m 16.5M - 18.0.0-3906-g1c956514 ace6904b066b d814852f934a 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mds.j smithi006 running (2m) 26s ago 2m 13.8M - 18.0.0-3906-g1c956514 ace6904b066b ed5730b914be 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mds.k smithi087 running (2m) 41s ago 2m 15.2M - 18.0.0-3906-g1c956514 ace6904b066b 5ef2c27fd963 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mds.l smithi186 running (2m) 41s ago 2m 17.3M - 18.0.0-3906-g1c956514 ace6904b066b 9351d0e622df 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.x smithi006 *:9283,8765 running (9m) 26s ago 9m 521M - 18.0.0-3906-g1c956514 ace6904b066b 31f93bc4dda3 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.y smithi087 *:8443,8765 running (7m) 41s ago 7m 447M - 18.0.0-3906-g1c956514 ace6904b066b 850a924c938f 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mgr.z smithi186 *:8443,8765 running (7m) 41s ago 7m 448M - 18.0.0-3906-g1c956514 ace6904b066b 1156cd27c013 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mon.a smithi006 running (9m) 26s ago 9m 96.7M 2048M 18.0.0-3906-g1c956514 ace6904b066b c70c8f5ff36c 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mon.b smithi087 running (7m) 41s ago 7m 63.6M 2048M 18.0.0-3906-g1c956514 ace6904b066b 0f17a802242f 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:mon.c smithi186 running (8m) 41s ago 8m 66.7M 2048M 18.0.0-3906-g1c956514 ace6904b066b cc7a1c1574c0 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 smithi006 running (7m) 26s ago 7m 71.0M 4319M 18.0.0-3906-g1c956514 ace6904b066b 5fb3ce343e81 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 smithi087 running (6m) 41s ago 6m 78.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 96649a25454c 2023-11-17T00:07:53.970 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 smithi186 running (6m) 41s ago 6m 77.4M 4292M 18.0.0-3906-g1c956514 ace6904b066b c89f6e2508c3 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 smithi006 running (6m) 26s ago 6m 65.5M 4319M 18.0.0-3906-g1c956514 ace6904b066b 092dbf957471 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 smithi087 running (5m) 41s ago 5m 63.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 76405edf90cf 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 smithi186 running (5m) 41s ago 5m 69.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 07c6c591a881 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 smithi006 running (4m) 26s ago 4m 60.0M 4319M 18.0.0-3906-g1c956514 ace6904b066b 1b2b4585e78f 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 smithi087 running (4m) 41s ago 4m 65.0M 4292M 18.0.0-3906-g1c956514 ace6904b066b 1cdea23d90f9 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 smithi186 running (4m) 41s ago 4m 61.4M 4292M 18.0.0-3906-g1c956514 ace6904b066b 269ab4ca8ec5 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 smithi006 running (3m) 26s ago 3m 60.9M 4319M 18.0.0-3906-g1c956514 ace6904b066b 6e0d8f4a146e 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 smithi087 running (3m) 41s ago 3m 59.9M 4292M 18.0.0-3906-g1c956514 ace6904b066b 69345b4974df 2023-11-17T00:07:53.971 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 smithi186 running (2m) 41s ago 2m 56.0M 4292M 18.0.0-3906-g1c956514 ace6904b066b d707e4afe838 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 msgr2=0x7feb340a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 0x7feb340a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb44101350 tx=0x7feb38009260 comp rx=0 tx=0).stop 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 msgr2=0x7feb4419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b070 tx=0x7feb3c00bce0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 shutdown_connections 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7feb340a1260 0x7feb340a3710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb44101e80 0x7feb4419f900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb441002f0 0x7feb44198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 --2- 172.21.15.6:0/540154478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb441014f0 0x7feb44198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:53.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 >> 172.21.15.6:0/540154478 conn(0x7feb440fb900 msgr2=0x7feb44106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:53.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 shutdown_connections 2023-11-17T00:07:53.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:53.968+0000 7feb4b1df700 1 -- 172.21.15.6:0/540154478 wait complete. 2023-11-17T00:07:54.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:54 smithi006 ceph-mon[97840]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:54.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:54 smithi006 ceph-mon[97840]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:54.687 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-11-17T00:07:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:54 smithi087 ceph-mon[131182]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:54 smithi087 ceph-mon[131182]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:54.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:54 smithi186 ceph-mon[131487]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:07:54.959 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:54 smithi186 ceph-mon[131487]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:55.553 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:07:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:56 smithi087 ceph-mon[131182]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:57.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:56 smithi186 ceph-mon[131487]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:57.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:56 smithi006 ceph-mon[97840]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:57.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.451+0000 7fbfd15f1700 1 -- 172.21.15.6:0/3833358518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc1002f0 msgr2=0x7fbfcc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.451+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/3833358518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc100700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009990 tx=0x7fbfbc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 -- 172.21.15.6:0/3833358518 shutdown_connections 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/3833358518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc100700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/3833358518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc101e80 0x7fbfcc108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/3833358518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 -- 172.21.15.6:0/3833358518 >> 172.21.15.6:0/3833358518 conn(0x7fbfcc0fb900 msgr2=0x7fbfcc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 -- 172.21.15.6:0/3833358518 shutdown_connections 2023-11-17T00:07:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.452+0000 7fbfd15f1700 1 -- 172.21.15.6:0/3833358518 wait complete. 2023-11-17T00:07:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 Processor -- start 2023-11-17T00:07:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 -- start start 2023-11-17T00:07:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19fe20 con 0x7fbfcc101e80 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19ff60 con 0x7fbfcc1002f0 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfd15f1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc1a00f0 con 0x7fbfcc1014f0 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.453+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36066/0 (socket says 172.21.15.6:36066) 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 -- 172.21.15.6:0/2228277363 learned_addr learned my addr 172.21.15.6:0/2228277363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcaffd700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfca7fc700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 -- 172.21.15.6:0/2228277363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 msgr2=0x7fbfcc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 -- 172.21.15.6:0/2228277363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1002f0 msgr2=0x7fbfcc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:57.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 -- 172.21.15.6:0/2228277363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0095f0 con 0x7fbfcc101e80 2023-11-17T00:07:57.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfca7fc700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfcb7fe700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400ef60 tx=0x7fbfc400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4018070 con 0x7fbfcc101e80 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc10bcb0 con 0x7fbfcc101e80 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.454+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc400f040 con 0x7fbfcc101e80 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.455+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400cd70 con 0x7fbfcc101e80 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.455+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc10bb20 con 0x7fbfcc101e80 2023-11-17T00:07:57.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.456+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfcc066ea0 con 0x7fbfcc101e80 2023-11-17T00:07:57.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.458+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fbfc4003a00 con 0x7fbfcc101e80 2023-11-17T00:07:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.459+0000 7fbfb3fff700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 0x7fbfb40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:07:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.459+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbfc40cc9a0 con 0x7fbfcc101e80 2023-11-17T00:07:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.459+0000 7fbfcaffd700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 0x7fbfb40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:07:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.460+0000 7fbfcaffd700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 0x7fbfb40a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b2e0 tx=0x7fbfbc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:07:57.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.463+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbfc4095cb0 con 0x7fbfcc101e80 2023-11-17T00:07:57.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.678+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfcc107e50 con 0x7fbfb40a11e0 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.684+0000 7fbfb3fff700 1 -- 172.21.15.6:0/2228277363 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fbfcc107e50 con 0x7fbfb40a11e0 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stdout:mds.all 12/12 44s ago 2m smithi006=a;smithi006=d;smithi006=g;smithi006=j;smithi087=b;smithi087=e;smithi087=h;smithi087=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stdout:mgr 3/3 44s ago 7m smithi006=x;smithi087=y;smithi186=z;count:3 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stdout:mon 3/3 44s ago 8m smithi006:172.21.15.6=a;smithi087:172.21.15.87=b;smithi186:172.21.15.186=c;count:3 2023-11-17T00:07:57.686 INFO:teuthology.orchestra.run.smithi006.stdout:osd 12 44s ago - 2023-11-17T00:07:57.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.686+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 msgr2=0x7fbfb40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:57.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.686+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 0x7fbfb40a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b2e0 tx=0x7fbfbc005fb0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.686+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 msgr2=0x7fbfcc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.686+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400ef60 tx=0x7fbfc400ef90 comp rx=0 tx=0).stop 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 shutdown_connections 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbfb40a11e0 0x7fbfb40a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfcc101e80 0x7fbfcc19f850 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfcc1002f0 0x7fbfcc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 --2- 172.21.15.6:0/2228277363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfcc1014f0 0x7fbfcc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 >> 172.21.15.6:0/2228277363 conn(0x7fbfcc0fb900 msgr2=0x7fbfcc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 shutdown_connections 2023-11-17T00:07:57.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:07:57.687+0000 7fbfd15f1700 1 -- 172.21.15.6:0/2228277363 wait complete. 2023-11-17T00:07:58.357 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-11-17T00:07:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:58 smithi087 ceph-mon[131182]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:07:58 smithi087 ceph-mon[131182]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:58 smithi186 ceph-mon[131487]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:07:58 smithi186 ceph-mon[131487]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:59.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:58 smithi006 ceph-mon[97840]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:07:59.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:07:58 smithi006 ceph-mon[97840]: pgmap v348: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:07:59.226 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:08:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:00 smithi087 ceph-mon[131182]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:00 smithi186 ceph-mon[131487]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 -- 172.21.15.6:0/2367359369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a81002f0 msgr2=0x7f73a8100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 --2- 172.21.15.6:0/2367359369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a81002f0 0x7f73a8100700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009990 tx=0x7f73a4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 -- 172.21.15.6:0/2367359369 shutdown_connections 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 --2- 172.21.15.6:0/2367359369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a81002f0 0x7f73a8100700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 --2- 172.21.15.6:0/2367359369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8101e80 0x7f73a8108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 --2- 172.21.15.6:0/2367359369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81014f0 0x7f73a8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.154+0000 7f73af3de700 1 -- 172.21.15.6:0/2367359369 >> 172.21.15.6:0/2367359369 conn(0x7f73a80fb900 msgr2=0x7f73a80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.155+0000 7f73af3de700 1 -- 172.21.15.6:0/2367359369 shutdown_connections 2023-11-17T00:08:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.155+0000 7f73af3de700 1 -- 172.21.15.6:0/2367359369 wait complete. 2023-11-17T00:08:01.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 Processor -- start 2023-11-17T00:08:01.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 -- start start 2023-11-17T00:08:01.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 0x7f73a8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a8101e80 0x7f73a8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8073ba0 con 0x7f73a8101e80 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8073d10 con 0x7f73a81014f0 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.156+0000 7f73af3de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a81a00b0 con 0x7f73a81002f0 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ac979700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 0x7f73a8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad97b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a8101e80 0x7f73a8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ac979700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 0x7f73a8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:54846/0 (socket says 172.21.15.6:54846) 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ac979700 1 -- 172.21.15.6:0/2547347361 learned_addr learned my addr 172.21.15.6:0/2547347361 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49366/0 (socket says 172.21.15.6:49366) 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 -- 172.21.15.6:0/2547347361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 msgr2=0x7f73a8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 0x7f73a8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 -- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a8101e80 msgr2=0x7f73a8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a8101e80 0x7f73a8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 -- 172.21.15.6:0/2547347361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40095f0 con 0x7f73a81002f0 2023-11-17T00:08:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.157+0000 7f73ad17a700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c00 tx=0x7f73a4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.158+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a401d070 con 0x7f73a81002f0 2023-11-17T00:08:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.158+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a810bc40 con 0x7f73a81002f0 2023-11-17T00:08:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.158+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a4022470 con 0x7f73a81002f0 2023-11-17T00:08:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.158+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4017650 con 0x7f73a81002f0 2023-11-17T00:08:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.158+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a810bab0 con 0x7f73a81002f0 2023-11-17T00:08:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.159+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a8066ea0 con 0x7f73a81002f0 2023-11-17T00:08:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.161+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f73a40225e0 con 0x7f73a81002f0 2023-11-17T00:08:01.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.162+0000 7f739e7fc700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 0x7f73940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:01.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.162+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f73a40ce350 con 0x7f73a81002f0 2023-11-17T00:08:01.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.163+0000 7f73ac979700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 0x7f73940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:01.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.163+0000 7f73ac979700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 0x7f73940a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d20 tx=0x7f7398009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:01.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.165+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73a4097660 con 0x7f73a81002f0 2023-11-17T00:08:01.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:00 smithi006 ceph-mon[97840]: pgmap v349: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:01.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.379+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f73a8060f30 con 0x7f73940a11e0 2023-11-17T00:08:01.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.380+0000 7f739e7fc700 1 -- 172.21.15.6:0/2547347361 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f73a8060f30 con 0x7f73940a11e0 2023-11-17T00:08:01.382 INFO:teuthology.orchestra.run.smithi006.stdout:HOST ADDR LABELS STATUS 2023-11-17T00:08:01.382 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 172.21.15.6 2023-11-17T00:08:01.383 INFO:teuthology.orchestra.run.smithi006.stdout:smithi087 172.21.15.87 2023-11-17T00:08:01.383 INFO:teuthology.orchestra.run.smithi006.stdout:smithi186 172.21.15.186 2023-11-17T00:08:01.383 INFO:teuthology.orchestra.run.smithi006.stdout:3 hosts in cluster 2023-11-17T00:08:01.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 msgr2=0x7f73940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 0x7f73940a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d20 tx=0x7f7398009260 comp rx=0 tx=0).stop 2023-11-17T00:08:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 msgr2=0x7f73a8074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:01.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c00 tx=0x7f73a4005720 comp rx=0 tx=0).stop 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 shutdown_connections 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f73940a11e0 0x7f73940a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73a8101e80 0x7f73a8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73a81002f0 0x7f73a8074890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 --2- 172.21.15.6:0/2547347361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a81014f0 0x7f73a8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 >> 172.21.15.6:0/2547347361 conn(0x7f73a80fb900 msgr2=0x7f73a80fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 shutdown_connections 2023-11-17T00:08:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:01.383+0000 7f73af3de700 1 -- 172.21.15.6:0/2547347361 wait complete. 2023-11-17T00:08:02.027 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-11-17T00:08:02.894 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:08:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:02 smithi087 ceph-mon[131182]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:02 smithi087 ceph-mon[131182]: pgmap v350: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-17T00:08:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-17T00:08:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:02 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-11-17T00:08:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:02 smithi186 ceph-mon[131487]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:02 smithi186 ceph-mon[131487]: pgmap v350: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-17T00:08:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-17T00:08:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:02 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-11-17T00:08:03.184 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:02 smithi006 ceph-mon[97840]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:03.184 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:02 smithi006 ceph-mon[97840]: pgmap v350: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:03.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-17T00:08:03.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-17T00:08:03.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:02 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-11-17T00:08:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:03 smithi186 ceph-mon[131487]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-11-17T00:08:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-17T00:08:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-17T00:08:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:03 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-11-17T00:08:04.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:03 smithi186 ceph-mon[131487]: osdmap e91: 12 total, 12 up, 12 in 2023-11-17T00:08:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:03 smithi006 ceph-mon[97840]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-11-17T00:08:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-17T00:08:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-17T00:08:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:03 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-11-17T00:08:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:03 smithi006 ceph-mon[97840]: osdmap e91: 12 total, 12 up, 12 in 2023-11-17T00:08:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:03 smithi087 ceph-mon[131182]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-11-17T00:08:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-17T00:08:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-17T00:08:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:03 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-11-17T00:08:04.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:03 smithi087 ceph-mon[131182]: osdmap e91: 12 total, 12 up, 12 in 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 -- 172.21.15.6:0/335339577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d28102510 msgr2=0x7f5d28102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/335339577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d28102510 0x7f5d28102920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009990 tx=0x7f5d1c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 -- 172.21.15.6:0/335339577 shutdown_connections 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/335339577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d28102510 0x7f5d28102920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/335339577 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d281040a0 0x7f5d2810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/335339577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 0x7f5d28103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 -- 172.21.15.6:0/335339577 >> 172.21.15.6:0/335339577 conn(0x7f5d280fdb60 msgr2=0x7f5d280fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:04.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 -- 172.21.15.6:0/335339577 shutdown_connections 2023-11-17T00:08:04.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.655+0000 7f5d2f10c700 1 -- 172.21.15.6:0/335339577 wait complete. 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.656+0000 7f5d2f10c700 1 Processor -- start 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.656+0000 7f5d2f10c700 1 -- start start 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 0x7f5d281987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 0x7f5d2819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:04.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2819fdd0 con 0x7f5d281040a0 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2819ff10 con 0x7f5d28103710 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2f10c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281a00a0 con 0x7f5d28102510 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2d6a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 0x7f5d2819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49382/0 (socket says 172.21.15.6:49382) 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 -- 172.21.15.6:0/1267443190 learned_addr learned my addr 172.21.15.6:0/1267443190 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d27fff700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 0x7f5d281987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 -- 172.21.15.6:0/1267443190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 msgr2=0x7f5d281987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 0x7f5d281987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 -- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 msgr2=0x7f5d2819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 0x7f5d2819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2cea8700 1 -- 172.21.15.6:0/1267443190 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0095f0 con 0x7f5d28102510 2023-11-17T00:08:04.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.657+0000 7f5d2d6a9700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 0x7f5d2819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d2cea8700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004740 tx=0x7f5d1c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c01d070 con 0x7f5d28102510 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2810deb0 con 0x7f5d28102510 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c021410 con 0x7f5d28102510 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017650 con 0x7f5d28102510 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.658+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2810dd20 con 0x7f5d28102510 2023-11-17T00:08:04.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.659+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d28109b30 con 0x7f5d28102510 2023-11-17T00:08:04.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.661+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5d1c021580 con 0x7f5d28102510 2023-11-17T00:08:04.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.663+0000 7f5d25ffb700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:04.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.663+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5d1c0ce090 con 0x7f5d28102510 2023-11-17T00:08:04.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.663+0000 7f5d27fff700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:04.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.663+0000 7f5d27fff700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 0x7f5d180a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009c60 tx=0x7f5d14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:04.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.667+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5d1c0974a0 con 0x7f5d28102510 2023-11-17T00:08:04.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.885+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d280619a0 con 0x7f5d180a1150 2023-11-17T00:08:04.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.887+0000 7f5d25ffb700 1 -- 172.21.15.6:0/1267443190 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f5d280619a0 con 0x7f5d180a1150 2023-11-17T00:08:04.904 INFO:teuthology.orchestra.run.smithi006.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-17T00:08:04.904 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330000H400BGN 372G No 3m ago LVM detected, locked 2023-11-17T00:08:04.904 INFO:teuthology.orchestra.run.smithi006.stdout:smithi087 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NQ400BGN 372G No 3m ago LVM detected, locked 2023-11-17T00:08:04.904 INFO:teuthology.orchestra.run.smithi006.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 2m ago LVM detected, locked 2023-11-17T00:08:04.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 msgr2=0x7f5d180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:04.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 0x7f5d180a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009c60 tx=0x7f5d14009260 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 msgr2=0x7f5d28198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004740 tx=0x7f5d1c009580 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 shutdown_connections 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d281040a0 0x7f5d2819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d28102510 0x7f5d28198290 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 --2- 172.21.15.6:0/1267443190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d28103710 0x7f5d281987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:05.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 >> 172.21.15.6:0/1267443190 conn(0x7f5d280fdb60 msgr2=0x7f5d28106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:05.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 shutdown_connections 2023-11-17T00:08:05.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:04.889+0000 7f5d2f10c700 1 -- 172.21.15.6:0/1267443190 wait complete. 2023-11-17T00:08:05.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:04 smithi186 ceph-mon[131487]: pgmap v352: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2023-11-17T00:08:05.177 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:04 smithi186 ceph-mon[131487]: osdmap e92: 12 total, 12 up, 12 in 2023-11-17T00:08:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:04 smithi006 ceph-mon[97840]: pgmap v352: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2023-11-17T00:08:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:04 smithi006 ceph-mon[97840]: osdmap e92: 12 total, 12 up, 12 in 2023-11-17T00:08:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:04 smithi087 ceph-mon[131182]: pgmap v352: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2023-11-17T00:08:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:04 smithi087 ceph-mon[131182]: osdmap e92: 12 total, 12 up, 12 in 2023-11-17T00:08:05.695 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-17T00:08:05.788 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:08:05.788 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-11-17T00:08:05.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:05 smithi006 ceph-mon[97840]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:06.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:05 smithi186 ceph-mon[131487]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:06.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:05 smithi087 ceph-mon[131182]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:08:07.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:06 smithi186 ceph-mon[131487]: pgmap v354: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:08:07.141 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:06 smithi006 ceph-mon[97840]: pgmap v354: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:08:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:06 smithi087 ceph-mon[131182]: pgmap v354: 129 pgs: 129 active+clean; 638 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 -- 172.21.15.6:0/1149343703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c1014f0 msgr2=0x7f861c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 --2- 172.21.15.6:0/1149343703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c1014f0 0x7f861c101940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f860c009990 tx=0x7f860c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 -- 172.21.15.6:0/1149343703 shutdown_connections 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 --2- 172.21.15.6:0/1149343703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c101e80 0x7f861c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 --2- 172.21.15.6:0/1149343703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 --2- 172.21.15.6:0/1149343703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c1014f0 0x7f861c101940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.542+0000 7f862466e700 1 -- 172.21.15.6:0/1149343703 >> 172.21.15.6:0/1149343703 conn(0x7f861c0fb900 msgr2=0x7f861c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.543+0000 7f862466e700 1 -- 172.21.15.6:0/1149343703 shutdown_connections 2023-11-17T00:08:08.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.543+0000 7f862466e700 1 -- 172.21.15.6:0/1149343703 wait complete. 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 Processor -- start 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 -- start start 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c1014f0 0x7f861c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c073ba0 con 0x7f861c101e80 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c073d10 con 0x7f861c1002f0 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862466e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c1a0130 con 0x7f861c1014f0 2023-11-17T00:08:08.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f8622c0b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862240a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.544+0000 7f862240a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:54886/0 (socket says 172.21.15.6:54886) 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34646/0 (socket says 172.21.15.6:34646) 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f862240a700 1 -- 172.21.15.6:0/3187973077 learned_addr learned my addr 172.21.15.6:0/3187973077 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8621c09700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c1014f0 0x7f861c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 -- 172.21.15.6:0/3187973077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c1014f0 msgr2=0x7f861c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c1014f0 0x7f861c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 -- 172.21.15.6:0/3187973077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 msgr2=0x7f861c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 -- 172.21.15.6:0/3187973077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0095f0 con 0x7f861c101e80 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f8622c0b700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f861400efc0 tx=0x7f861400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614018070 con 0x7f861c101e80 2023-11-17T00:08:08.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861400f040 con 0x7f861c101e80 2023-11-17T00:08:08.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861400cd40 con 0x7f861c101e80 2023-11-17T00:08:08.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.545+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c10bc90 con 0x7f861c101e80 2023-11-17T00:08:08.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.546+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c10bb00 con 0x7f861c101e80 2023-11-17T00:08:08.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.547+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c1078e0 con 0x7f861c101e80 2023-11-17T00:08:08.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.549+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8614003a00 con 0x7f861c101e80 2023-11-17T00:08:08.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.550+0000 7f86137fe700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 0x7f86080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:08.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.550+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f86140cc7d0 con 0x7f861c101e80 2023-11-17T00:08:08.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.550+0000 7f862240a700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 0x7f86080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:08.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.551+0000 7f862240a700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 0x7f86080a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8618005950 tx=0x7f8618009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:08.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.554+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8614095be0 con 0x7f861c101e80 2023-11-17T00:08:08.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.804+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f861c02cfe0 con 0x7f861c101e80 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.804+0000 7f86137fe700 1 -- 172.21.15.6:0/3187973077 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2857 (secure 0 0 0) 0x7f8614099200 con 0x7f861c101e80 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout:e20 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout:legacy client fscid: 1 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:08.806 INFO:teuthology.orchestra.run.smithi006.stdout:Filesystem 'cephfs' (1) 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:fs_name cephfs 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 20 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:created 2023-11-17T00:06:12.598183+0000 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2023-11-17T00:06:37.833900+0000 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:tableserver 0 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:root 0 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:session_timeout 300 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:session_autoclose 300 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:max_file_size 1099511627776 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:max_xattr_size 65536 2023-11-17T00:08:08.807 INFO:teuthology.orchestra.run.smithi006.stdout:required_client_features {} 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure 0 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:last_failure_osd_epoch 0 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:max_mds 1 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:in 0 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:up {0=24578} 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:failed 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:damaged 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:stopped 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:data_pools [3,4] 2023-11-17T00:08:08.808 INFO:teuthology.orchestra.run.smithi006.stdout:metadata_pool 2 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:inline_data disabled 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:balancer 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:bal_rank_mask -1 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:standby_count_wanted 0 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b{0:24578} state up:active seq 5 addr [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.h{0:24539} state up:standby-replay seq 1 addr [v2:172.21.15.87:6836/850260336,v1:172.21.15.87:6837/850260336] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:Standby daemons: 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.j{-1:14442} state up:standby seq 1 addr [v2:172.21.15.6:6834/3779928633,v1:172.21.15.6:6835/3779928633] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d{-1:14454} state up:standby seq 1 addr [v2:172.21.15.6:6836/3190561786,v1:172.21.15.6:6837/3190561786] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.809 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.g{-1:14508} state up:standby seq 1 addr [v2:172.21.15.6:6838/3324637103,v1:172.21.15.6:6839/3324637103] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a{-1:14514} state up:standby seq 1 addr [v2:172.21.15.6:6840/1535787123,v1:172.21.15.6:6841/1535787123] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.186:6832/3460267668,v1:172.21.15.186:6833/3460267668] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.87:6832/2959279613,v1:172.21.15.87:6833/2959279613] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.f{-1:24499} state up:standby seq 1 addr [v2:172.21.15.186:6834/1004385520,v1:172.21.15.186:6835/1004385520] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.e{-1:24509} state up:standby seq 1 addr [v2:172.21.15.87:6834/2088255632,v1:172.21.15.87:6835/2088255632] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c{-1:24511} state up:standby seq 1 addr [v2:172.21.15.186:6836/3856911296,v1:172.21.15.186:6837/3856911296] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.810 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.i{-1:24532} state up:standby seq 1 addr [v2:172.21.15.186:6838/143156596,v1:172.21.15.186:6839/143156596] compat {c=[1],r=[1],i=[7ff]}] 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 msgr2=0x7f86080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 0x7f86080a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8618005950 tx=0x7f8618009260 comp rx=0 tx=0).stop 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 msgr2=0x7f861c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f861400efc0 tx=0x7f861400c3a0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 shutdown_connections 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f86080a11d0 0x7f86080a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f861c101e80 0x7f861c073420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f861c1002f0 0x7f861c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 --2- 172.21.15.6:0/3187973077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f861c1014f0 0x7f861c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:08.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 >> 172.21.15.6:0/3187973077 conn(0x7f861c0fb900 msgr2=0x7f861c104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:08.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 shutdown_connections 2023-11-17T00:08:08.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:08.806+0000 7f862466e700 1 -- 172.21.15.6:0/3187973077 wait complete. 2023-11-17T00:08:08.813 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:08:09.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:08 smithi006 ceph-mon[97840]: pgmap v355: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2023-11-17T00:08:09.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:08 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3187973077' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-17T00:08:09.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:08 smithi087 ceph-mon[131182]: pgmap v355: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2023-11-17T00:08:09.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:08 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3187973077' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-17T00:08:09.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:08 smithi186 ceph-mon[131487]: pgmap v355: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2023-11-17T00:08:09.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:08 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3187973077' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-17T00:08:09.558 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-11-17T00:08:10.419 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:08:11.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:10 smithi006 ceph-mon[97840]: pgmap v356: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-11-17T00:08:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:10 smithi087 ceph-mon[131182]: pgmap v356: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-11-17T00:08:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:10 smithi186 ceph-mon[131487]: pgmap v356: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 -- 172.21.15.6:0/3566206224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd8101e80 msgr2=0x7f2cd8108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/3566206224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd8101e80 0x7f2cd8108740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000b0a0 tx=0x7f2cd000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 -- 172.21.15.6:0/3566206224 shutdown_connections 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/3566206224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd8101e80 0x7f2cd8108740 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/3566206224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 0x7f2cd8100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/3566206224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd81014f0 0x7f2cd8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 -- 172.21.15.6:0/3566206224 >> 172.21.15.6:0/3566206224 conn(0x7f2cd80fb900 msgr2=0x7f2cd80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:12.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 -- 172.21.15.6:0/3566206224 shutdown_connections 2023-11-17T00:08:12.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.427+0000 7f2cdf7da700 1 -- 172.21.15.6:0/3566206224 wait complete. 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.428+0000 7f2cdf7da700 1 Processor -- start 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.428+0000 7f2cdf7da700 1 -- start start 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 0x7f2cd81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd8101e80 0x7f2cd819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd819fe80 con 0x7f2cd81014f0 2023-11-17T00:08:12.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd819ffc0 con 0x7f2cd8101e80 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdf7da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81a0150 con 0x7f2cd81002f0 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdd576700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 0x7f2cd81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59654/0 (socket says 172.21.15.6:59654) 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 -- 172.21.15.6:0/2836779360 learned_addr learned my addr 172.21.15.6:0/2836779360 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cddd77700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd8101e80 0x7f2cd819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 -- 172.21.15.6:0/2836779360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 msgr2=0x7f2cd81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 0x7f2cd81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 -- 172.21.15.6:0/2836779360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd8101e80 msgr2=0x7f2cd819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:12.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd8101e80 0x7f2cd819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.429+0000 7f2cdcd75700 1 -- 172.21.15.6:0/2836779360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0009d00 con 0x7f2cd81014f0 2023-11-17T00:08:12.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cdcd75700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800e9d0 tx=0x7f2cc800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:12.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8009790 con 0x7f2cd81014f0 2023-11-17T00:08:12.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc800ee70 con 0x7f2cd81014f0 2023-11-17T00:08:12.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd810bc50 con 0x7f2cd81014f0 2023-11-17T00:08:12.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8010ea0 con 0x7f2cd81014f0 2023-11-17T00:08:12.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.430+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd810bac0 con 0x7f2cd81014f0 2023-11-17T00:08:12.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.431+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd81a0880 con 0x7f2cd81014f0 2023-11-17T00:08:12.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.433+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2cc800cbd0 con 0x7f2cd81014f0 2023-11-17T00:08:12.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.435+0000 7f2cce7fc700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 0x7f2cc40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:12.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.435+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2cc8014070 con 0x7f2cd81014f0 2023-11-17T00:08:12.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.435+0000 7f2cdd576700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 0x7f2cc40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.435+0000 7f2cdd576700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 0x7f2cc40a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8101350 tx=0x7f2cd400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:12.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.438+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2cc8051730 con 0x7f2cd81014f0 2023-11-17T00:08:12.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.650+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f2cd81a0e30 con 0x7f2cd81014f0 2023-11-17T00:08:12.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.651+0000 7f2cce7fc700 1 -- 172.21.15.6:0/2836779360 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5640 (secure 0 0 0) 0x7f2cc80961a0 con 0x7f2cd81014f0 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 92 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:created 2023-11-16T23:58:00.220236+0000 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2023-11-17T00:08:03.838535+0000 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:crush_version 27 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:full_ratio 0.95 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:backfillfull_ratio 0.9 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:nearfull_ratio 0.85 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:require_min_compat_client luminous 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:min_compat_client luminous 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:require_osd_release reef 2023-11-17T00:08:12.653 INFO:teuthology.orchestra.run.smithi006.stdout:stretch_mode_enabled false 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:max_osd 12 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6802/3863583460,v1:172.21.15.6:6803/3863583460] [v2:172.21.15.6:6804/3863583460,v1:172.21.15.6:6805/3863583460] exists,up bf2da7d9-d6af-4949-9a5f-d392c2dcc273 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6800/4270277324,v1:172.21.15.87:6801/4270277324] [v2:172.21.15.87:6802/4270277324,v1:172.21.15.87:6803/4270277324] exists,up cbdc1ecd-4466-47fc-8510-60d676144e56 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/1950482303,v1:172.21.15.186:6801/1950482303] [v2:172.21.15.186:6802/1950482303,v1:172.21.15.186:6803/1950482303] exists,up 17362d0b-f379-4a0e-a815-aec6d56732e4 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6810/1124657350,v1:172.21.15.6:6811/1124657350] [v2:172.21.15.6:6812/1124657350,v1:172.21.15.6:6813/1124657350] exists,up d94de1b8-5e3a-4344-b794-04389000f0d6 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 up in weight 1 up_from 33 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6808/31913471,v1:172.21.15.87:6809/31913471] [v2:172.21.15.87:6810/31913471,v1:172.21.15.87:6811/31913471] exists,up 17ec1e9b-999c-47e6-87ca-3405277892f1 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/2177950803,v1:172.21.15.186:6809/2177950803] [v2:172.21.15.186:6810/2177950803,v1:172.21.15.186:6811/2177950803] exists,up f8e5ab3b-1735-4f77-82c8-a374f7fab368 2023-11-17T00:08:12.654 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6818/1113025990,v1:172.21.15.6:6819/1113025990] [v2:172.21.15.6:6820/1113025990,v1:172.21.15.6:6821/1113025990] exists,up 8efcc3e3-0f96-4248-8eac-6f5041c0b4fb 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6816/1542327799,v1:172.21.15.87:6817/1542327799] [v2:172.21.15.87:6818/1542327799,v1:172.21.15.87:6819/1542327799] exists,up 9282e04d-a458-456a-ac74-a942aa06cdc5 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:osd.8 up in weight 1 up_from 58 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/4126523122,v1:172.21.15.186:6817/4126523122] [v2:172.21.15.186:6818/4126523122,v1:172.21.15.186:6819/4126523122] exists,up 02359124-89e5-4f4a-a547-b5fe9e03d316 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:osd.9 up in weight 1 up_from 66 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.6:6826/1254890156,v1:172.21.15.6:6827/1254890156] [v2:172.21.15.6:6828/1254890156,v1:172.21.15.6:6829/1254890156] exists,up 75b7d811-9e00-427c-a0b7-e300fd19a9df 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:osd.10 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6824/1132375941,v1:172.21.15.87:6825/1132375941] [v2:172.21.15.87:6826/1132375941,v1:172.21.15.87:6827/1132375941] exists,up 5acfeda0-5332-4020-bcc7-2173606e96ad 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/3407026982,v1:172.21.15.186:6825/3407026982] [v2:172.21.15.186:6826/3407026982,v1:172.21.15.186:6827/3407026982] exists,up f058968e-7426-4a1c-917e-09beb2be51ac 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.0 [8,4] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.a [2,11] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.e [2,4] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 2.13 [2,11] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.13 [4,9] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.17 [4,6] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.1e [3,9] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.21 [4,8] 2023-11-17T00:08:12.655 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 3.25 [5,9] 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.8 [5,8] 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.b [9,8] 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.14 [9,8] 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_upmap_items 4.1f [9,10] 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/1821215069 expires 2023-11-17T23:59:01.653365+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/4157530954 expires 2023-11-17T23:59:01.653365+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/610057204 expires 2023-11-17T23:59:01.653365+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6800/3715783566 expires 2023-11-17T23:58:33.059992+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3407461840 expires 2023-11-17T23:58:33.059992+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3069406217 expires 2023-11-17T23:58:33.059992+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/3715783566 expires 2023-11-17T23:58:33.059992+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:6801/610057204 expires 2023-11-17T23:59:01.653365+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/3467557048 expires 2023-11-17T23:58:33.059992+0000 2023-11-17T00:08:12.656 INFO:teuthology.orchestra.run.smithi006.stdout:blocklist 172.21.15.6:0/2971625275 expires 2023-11-17T23:59:01.653365+0000 2023-11-17T00:08:12.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 msgr2=0x7f2cc40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:12.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 0x7f2cc40a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8101350 tx=0x7f2cd400b410 comp rx=0 tx=0).stop 2023-11-17T00:08:12.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 msgr2=0x7f2cd8198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:12.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800e9d0 tx=0x7f2cc800ece0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 shutdown_connections 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f2cc40a11d0 0x7f2cc40a3680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd8101e80 0x7f2cd819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd81002f0 0x7f2cd81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 --2- 172.21.15.6:0/2836779360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cd81014f0 0x7f2cd8198830 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 >> 172.21.15.6:0/2836779360 conn(0x7f2cd80fb900 msgr2=0x7f2cd80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 shutdown_connections 2023-11-17T00:08:12.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:12.653+0000 7f2cdf7da700 1 -- 172.21.15.6:0/2836779360 wait complete. 2023-11-17T00:08:12.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:12 smithi006 ceph-mon[97840]: pgmap v357: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 344 B/s wr, 2 op/s 2023-11-17T00:08:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:12 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2836779360' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-17T00:08:13.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:12 smithi087 ceph-mon[131182]: pgmap v357: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 344 B/s wr, 2 op/s 2023-11-17T00:08:13.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:12 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2836779360' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-17T00:08:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:12 smithi186 ceph-mon[131487]: pgmap v357: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 344 B/s wr, 2 op/s 2023-11-17T00:08:13.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:12 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2836779360' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-17T00:08:13.427 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-17T00:08:13.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:13.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 -- 172.21.15.6:0/3001316183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081014c0 msgr2=0x7fed08101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:13.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/3001316183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081014c0 0x7fed08101910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009990 tx=0x7fecfc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 -- 172.21.15.6:0/3001316183 shutdown_connections 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/3001316183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 0x7fed08108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/3001316183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081002d0 0x7fed081006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/3001316183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081014c0 0x7fed08101910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 -- 172.21.15.6:0/3001316183 >> 172.21.15.6:0/3001316183 conn(0x7fed080fb8b0 msgr2=0x7fed080fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 -- 172.21.15.6:0/3001316183 shutdown_connections 2023-11-17T00:08:13.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.783+0000 7fed0e1ff700 1 -- 172.21.15.6:0/3001316183 wait complete. 2023-11-17T00:08:13.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 Processor -- start 2023-11-17T00:08:13.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 -- start start 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 0x7fed08196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 0x7fed0819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819dbb0 con 0x7fed081002d0 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819dcf0 con 0x7fed081014c0 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.784+0000 7fed0e1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0819de80 con 0x7fed08101e50 2023-11-17T00:08:13.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 0x7fed08196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 0x7fed08196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59666/0 (socket says 172.21.15.6:59666) 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52482/0 (socket says 172.21.15.6:52482) 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed077fe700 1 -- 172.21.15.6:0/4099179256 learned_addr learned my addr 172.21.15.6:0/4099179256 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed07fff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 0x7fed0819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 -- 172.21.15.6:0/4099179256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 msgr2=0x7fed0819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 0x7fed0819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 -- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 msgr2=0x7fed08196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 0x7fed08196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 -- 172.21.15.6:0/4099179256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0095f0 con 0x7fed081014c0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed06ffd700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b620 tx=0x7fecfc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.785+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01d070 con 0x7fed081014c0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.786+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0810bc80 con 0x7fed081014c0 2023-11-17T00:08:13.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.786+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecfc021410 con 0x7fed081014c0 2023-11-17T00:08:13.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.786+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc017650 con 0x7fed081014c0 2023-11-17T00:08:13.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.786+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0810baf0 con 0x7fed081014c0 2023-11-17T00:08:13.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.787+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed08067af0 con 0x7fed081014c0 2023-11-17T00:08:13.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.789+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fecfc021580 con 0x7fed081014c0 2023-11-17T00:08:13.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.790+0000 7fed04ff9700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 0x7fecf00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:13.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.790+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fecfc0ce1e0 con 0x7fed081014c0 2023-11-17T00:08:13.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.790+0000 7fed077fe700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 0x7fecf00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:13.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.791+0000 7fed077fe700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 0x7fecf00a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005950 tx=0x7fecf8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:13.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:13.792+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fecfc0975f0 con 0x7fed081014c0 2023-11-17T00:08:14.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.008+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed08107160 con 0x7fed081014c0 2023-11-17T00:08:14.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.011+0000 7fed04ff9700 1 -- 172.21.15.6:0/4099179256 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7fecfc09ac10 con 0x7fed081014c0 2023-11-17T00:08:14.012 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:14.012 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 msgr2=0x7fecf00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 0x7fecf00a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005950 tx=0x7fecf8009260 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 msgr2=0x7fed08196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b620 tx=0x7fecfc005740 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 shutdown_connections 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fecf00a11d0 0x7fecf00a3680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed081002d0 0x7fed08196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed08101e50 0x7fed0819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 --2- 172.21.15.6:0/4099179256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed081014c0 0x7fed08196570 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 >> 172.21.15.6:0/4099179256 conn(0x7fed080fb8b0 msgr2=0x7fed080fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:14.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.012+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 shutdown_connections 2023-11-17T00:08:14.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.013+0000 7fed0e1ff700 1 -- 172.21.15.6:0/4099179256 wait complete. 2023-11-17T00:08:14.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:08:14.034 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:08:14.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 -- 172.21.15.6:0/1489322601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4101480 msgr2=0x7fe5f41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/1489322601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4101480 0x7fe5f41018d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009990 tx=0x7fe5e8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 -- 172.21.15.6:0/1489322601 shutdown_connections 2023-11-17T00:08:14.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/1489322601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 0x7fe5f41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/1489322601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4100290 0x7fe5f41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/1489322601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4101480 0x7fe5f41018d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 -- 172.21.15.6:0/1489322601 >> 172.21.15.6:0/1489322601 conn(0x7fe5f40fb870 msgr2=0x7fe5f40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 -- 172.21.15.6:0/1489322601 shutdown_connections 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.242+0000 7fe5f9fff700 1 -- 172.21.15.6:0/1489322601 wait complete. 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.243+0000 7fe5f9fff700 1 Processor -- start 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 -- start start 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4101480 0x7fe5f4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 0x7fe5f419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f419fd60 con 0x7fe5f4100290 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f419fea0 con 0x7fe5f4101480 2023-11-17T00:08:14.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f9fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41a0030 con 0x7fe5f4101e10 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59682/0 (socket says 172.21.15.6:59682) 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f37fe700 1 -- 172.21.15.6:0/3203428558 learned_addr learned my addr 172.21.15.6:0/3203428558 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f2ffd700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4101480 0x7fe5f4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.244+0000 7fe5f3fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 0x7fe5f419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 -- 172.21.15.6:0/3203428558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 msgr2=0x7fe5f419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 0x7fe5f419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 -- 172.21.15.6:0/3203428558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4101480 msgr2=0x7fe5f4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4101480 0x7fe5f4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 -- 172.21.15.6:0/3203428558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80095f0 con 0x7fe5f4100290 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f37fe700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e970 tx=0x7fe5e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4009790 con 0x7fe5f4100290 2023-11-17T00:08:14.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e400c7e0 con 0x7fe5f4100290 2023-11-17T00:08:14.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4003c70 con 0x7fe5f4100290 2023-11-17T00:08:14.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f410bc50 con 0x7fe5f4100290 2023-11-17T00:08:14.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.245+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f410bac0 con 0x7fe5f4100290 2023-11-17T00:08:14.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.246+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f41a0730 con 0x7fe5f4100290 2023-11-17T00:08:14.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.248+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe5e400cc50 con 0x7fe5f4100290 2023-11-17T00:08:14.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.250+0000 7fe5f0ff9700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 0x7fe5dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.250+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe5e4014070 con 0x7fe5f4100290 2023-11-17T00:08:14.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.250+0000 7fe5f2ffd700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 0x7fe5dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.251+0000 7fe5f2ffd700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 0x7fe5dc0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b2e0 tx=0x7fe5e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.252+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe5e4096370 con 0x7fe5f4100290 2023-11-17T00:08:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.463+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5f41a0ce0 con 0x7fe5f4100290 2023-11-17T00:08:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.464+0000 7fe5f0ff9700 1 -- 172.21.15.6:0/3203428558 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5e4099990 con 0x7fe5f4100290 2023-11-17T00:08:14.465 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 msgr2=0x7fe5dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 0x7fe5dc0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b2e0 tx=0x7fe5e8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 msgr2=0x7fe5f41981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e970 tx=0x7fe5e400ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 shutdown_connections 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe5dc0a1150 0x7fe5dc0a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5f4100290 0x7fe5f41981e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5f4101e10 0x7fe5f419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 --2- 172.21.15.6:0/3203428558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5f4101480 0x7fe5f4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 >> 172.21.15.6:0/3203428558 conn(0x7fe5f40fb870 msgr2=0x7fe5f40fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 shutdown_connections 2023-11-17T00:08:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.466+0000 7fe5f9fff700 1 -- 172.21.15.6:0/3203428558 wait complete. 2023-11-17T00:08:14.487 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 -- 172.21.15.6:0/1510705514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 msgr2=0x7fbea41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 --2- 172.21.15.6:0/1510705514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea41006a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009990 tx=0x7fbe94009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 -- 172.21.15.6:0/1510705514 shutdown_connections 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 --2- 172.21.15.6:0/1510705514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea41006a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 --2- 172.21.15.6:0/1510705514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 0x7fbea41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.691+0000 7fbea8bff700 1 --2- 172.21.15.6:0/1510705514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 0x7fbea41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.692+0000 7fbea8bff700 1 -- 172.21.15.6:0/1510705514 >> 172.21.15.6:0/1510705514 conn(0x7fbea40fb870 msgr2=0x7fbea40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.692+0000 7fbea8bff700 1 -- 172.21.15.6:0/1510705514 shutdown_connections 2023-11-17T00:08:14.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.692+0000 7fbea8bff700 1 -- 172.21.15.6:0/1510705514 wait complete. 2023-11-17T00:08:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.693+0000 7fbea8bff700 1 Processor -- start 2023-11-17T00:08:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.693+0000 7fbea8bff700 1 -- start start 2023-11-17T00:08:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 0x7fbea4196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 0x7fbea419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea419db70 con 0x7fbea4100290 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea419dcb0 con 0x7fbea4101e10 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea8bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea419de40 con 0x7fbea4101480 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59710/0 (socket says 172.21.15.6:59710) 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 -- 172.21.15.6:0/721822387 learned_addr learned my addr 172.21.15.6:0/721822387 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea2d9d700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 0x7fbea419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea1d9b700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 0x7fbea4196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 -- 172.21.15.6:0/721822387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 msgr2=0x7fbea4196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 0x7fbea4196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 -- 172.21.15.6:0/721822387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 msgr2=0x7fbea419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.694+0000 7fbea259c700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 0x7fbea419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbea259c700 1 -- 172.21.15.6:0/721822387 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940095f0 con 0x7fbea4100290 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbea259c700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b620 tx=0x7fbe94005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401d070 con 0x7fbea4100290 2023-11-17T00:08:14.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe94021410 con 0x7fbea4100290 2023-11-17T00:08:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea410bc40 con 0x7fbea4100290 2023-11-17T00:08:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94017650 con 0x7fbea4100290 2023-11-17T00:08:14.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.695+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea410bab0 con 0x7fbea4100290 2023-11-17T00:08:14.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.696+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4107950 con 0x7fbea4100290 2023-11-17T00:08:14.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.698+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbe94021580 con 0x7fbea4100290 2023-11-17T00:08:14.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.699+0000 7fbe937fe700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 0x7fbe8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:14.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.700+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbe940ce1b0 con 0x7fbea4100290 2023-11-17T00:08:14.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.700+0000 7fbea1d9b700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 0x7fbe8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:14.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.700+0000 7fbea1d9b700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 0x7fbe8c0a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe98007c60 tx=0x7fbe980073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:14.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.703+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe940975c0 con 0x7fbea4100290 2023-11-17T00:08:14.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:14 smithi006 ceph-mon[97840]: pgmap v358: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s 2023-11-17T00:08:14.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:14 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4099179256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:14 smithi087 ceph-mon[131182]: pgmap v358: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s 2023-11-17T00:08:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:14 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4099179256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:14 smithi186 ceph-mon[131487]: pgmap v358: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 B/s wr, 2 op/s 2023-11-17T00:08:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:14 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4099179256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:14.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.913+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbea4061a50 con 0x7fbea4100290 2023-11-17T00:08:14.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.913+0000 7fbe937fe700 1 -- 172.21.15.6:0/721822387 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe9409abe0 con 0x7fbea4100290 2023-11-17T00:08:14.915 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 msgr2=0x7fbe8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 0x7fbe8c0a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe98007c60 tx=0x7fbe980073d0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 msgr2=0x7fbea4196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b620 tx=0x7fbe94005740 comp rx=0 tx=0).stop 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 shutdown_connections 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbe8c0a1150 0x7fbe8c0a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbea4100290 0x7fbea4196040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea4101e10 0x7fbea419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 --2- 172.21.15.6:0/721822387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4101480 0x7fbea4196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 >> 172.21.15.6:0/721822387 conn(0x7fbea40fb870 msgr2=0x7fbea4104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 shutdown_connections 2023-11-17T00:08:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:14.916+0000 7fbea8bff700 1 -- 172.21.15.6:0/721822387 wait complete. 2023-11-17T00:08:14.937 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:08:15.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 -- 172.21.15.6:0/4156496520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414101480 msgr2=0x7f54141018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4156496520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414101480 0x7f54141018d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5404009990 tx=0x7f5404009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 -- 172.21.15.6:0/4156496520 shutdown_connections 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4156496520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101e10 0x7f54141086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4156496520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414100290 0x7f54141006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4156496520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414101480 0x7f54141018d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 -- 172.21.15.6:0/4156496520 >> 172.21.15.6:0/4156496520 conn(0x7f54140fb870 msgr2=0x7f54140fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 -- 172.21.15.6:0/4156496520 shutdown_connections 2023-11-17T00:08:15.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.143+0000 7f541b7ff700 1 -- 172.21.15.6:0/4156496520 wait complete. 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.144+0000 7f541b7ff700 1 Processor -- start 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 -- start start 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 0x7f5414072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 0x7f5414073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414073bc0 con 0x7f5414100290 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414073d30 con 0x7f5414101480 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54141a0030 con 0x7f5414101e10 2023-11-17T00:08:15.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541959b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541959b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59732/0 (socket says 172.21.15.6:59732) 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f5418d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 0x7f5414072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f541959b700 1 -- 172.21.15.6:0/4145327208 learned_addr learned my addr 172.21.15.6:0/4145327208 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.145+0000 7f5419d9c700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 0x7f5414073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 -- 172.21.15.6:0/4145327208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 msgr2=0x7f5414073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 0x7f5414073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 -- 172.21.15.6:0/4145327208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 msgr2=0x7f5414072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 0x7f5414072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 -- 172.21.15.6:0/4145327208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040095f0 con 0x7f5414100290 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f5418d9a700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 0x7f5414072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f5419d9c700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 0x7f5414073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541959b700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f541000e970 tx=0x7f541000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:15.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410009790 con 0x7f5414100290 2023-11-17T00:08:15.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541000eec0 con 0x7f5414100290 2023-11-17T00:08:15.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410003cf0 con 0x7f5414100290 2023-11-17T00:08:15.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541410bc50 con 0x7f5414100290 2023-11-17T00:08:15.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.146+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541410bac0 con 0x7f5414100290 2023-11-17T00:08:15.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.147+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5414107210 con 0x7f5414100290 2023-11-17T00:08:15.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.149+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f541000ccd0 con 0x7f5414100290 2023-11-17T00:08:15.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.151+0000 7f540a7fc700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 0x7f54000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.151+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5410014070 con 0x7f5414100290 2023-11-17T00:08:15.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.151+0000 7f5418d9a700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 0x7f54000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.152+0000 7f5418d9a700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 0x7f54000a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f540400b2e0 tx=0x7f5404005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:15.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.154+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5410096210 con 0x7f5414100290 2023-11-17T00:08:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.365+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5414061700 con 0x7f5414100290 2023-11-17T00:08:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.365+0000 7f540a7fc700 1 -- 172.21.15.6:0/4145327208 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5410099830 con 0x7f5414100290 2023-11-17T00:08:15.367 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:08:15.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 msgr2=0x7f54000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 0x7f54000a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f540400b2e0 tx=0x7f5404005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 msgr2=0x7f54140748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f541000e970 tx=0x7f541000ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 shutdown_connections 2023-11-17T00:08:15.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54000a11d0 0x7f54000a3680 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5414100290 0x7f54140748a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5414101e10 0x7f5414073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 --2- 172.21.15.6:0/4145327208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5414101480 0x7f5414072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 >> 172.21.15.6:0/4145327208 conn(0x7f54140fb870 msgr2=0x7f54140fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 shutdown_connections 2023-11-17T00:08:15.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.368+0000 7f541b7ff700 1 -- 172.21.15.6:0/4145327208 wait complete. 2023-11-17T00:08:15.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:08:15.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 -- 172.21.15.6:0/4181283025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 msgr2=0x7fb8f4073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 --2- 172.21.15.6:0/4181283025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4073930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009990 tx=0x7fb8e8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 -- 172.21.15.6:0/4181283025 shutdown_connections 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 --2- 172.21.15.6:0/4181283025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4107ed0 0x7fb8f410a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 --2- 172.21.15.6:0/4181283025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4074a90 0x7fb8f4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 --2- 172.21.15.6:0/4181283025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4073930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 -- 172.21.15.6:0/4181283025 >> 172.21.15.6:0/4181283025 conn(0x7fb8f40fb650 msgr2=0x7fb8f40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.594+0000 7fb8f919d700 1 -- 172.21.15.6:0/4181283025 shutdown_connections 2023-11-17T00:08:15.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.595+0000 7fb8f919d700 1 -- 172.21.15.6:0/4181283025 wait complete. 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.595+0000 7fb8f919d700 1 Processor -- start 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 -- start start 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4074a90 0x7fb8f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4107ed0 0x7fb8f419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f419fb80 con 0x7fb8f40734c0 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f419fcc0 con 0x7fb8f4074a90 2023-11-17T00:08:15.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f919d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f419fe50 con 0x7fb8f4107ed0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59760/0 (socket says 172.21.15.6:59760) 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4074a90 0x7fb8f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f3fff700 1 -- 172.21.15.6:0/163508396 learned_addr learned my addr 172.21.15.6:0/163508396 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.596+0000 7fb8f899c700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4107ed0 0x7fb8f419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 -- 172.21.15.6:0/163508396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4107ed0 msgr2=0x7fb8f419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4107ed0 0x7fb8f419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 -- 172.21.15.6:0/163508396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4074a90 msgr2=0x7fb8f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4074a90 0x7fb8f4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 -- 172.21.15.6:0/163508396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80095f0 con 0x7fb8f40734c0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f3fff700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e970 tx=0x7fb8e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4009790 con 0x7fb8f40734c0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e400c7e0 con 0x7fb8f40734c0 2023-11-17T00:08:15.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4003c70 con 0x7fb8f40734c0 2023-11-17T00:08:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f410dc30 con 0x7fb8f40734c0 2023-11-17T00:08:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.597+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f410daa0 con 0x7fb8f40734c0 2023-11-17T00:08:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.598+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4106660 con 0x7fb8f40734c0 2023-11-17T00:08:15.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.600+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb8e400cc50 con 0x7fb8f40734c0 2023-11-17T00:08:15.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.601+0000 7fb8f17fa700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 0x7fb8dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.603+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb8e4014070 con 0x7fb8f40734c0 2023-11-17T00:08:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.603+0000 7fb8f37fe700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 0x7fb8dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:15.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.603+0000 7fb8f37fe700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 0x7fb8dc0a3680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8006010 tx=0x7fb8e8005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:15.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.605+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb8e4096170 con 0x7fb8f40734c0 2023-11-17T00:08:15.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3203428558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/721822387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4145327208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.813+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb8f4061700 con 0x7fb8f40734c0 2023-11-17T00:08:15.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.814+0000 7fb8f17fa700 1 -- 172.21.15.6:0/163508396 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb8e4003dd0 con 0x7fb8f40734c0 2023-11-17T00:08:15.815 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:15.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 msgr2=0x7fb8dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 0x7fb8dc0a3680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8006010 tx=0x7fb8e8005a90 comp rx=0 tx=0).stop 2023-11-17T00:08:15.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 msgr2=0x7fb8f4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e970 tx=0x7fb8e400ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 shutdown_connections 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fb8dc0a11d0 0x7fb8dc0a3680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8f40734c0 0x7fb8f4198000 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8f4107ed0 0x7fb8f419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 --2- 172.21.15.6:0/163508396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8f4074a90 0x7fb8f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 >> 172.21.15.6:0/163508396 conn(0x7fb8f40fb650 msgr2=0x7fb8f40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.816+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 shutdown_connections 2023-11-17T00:08:15.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:15.817+0000 7fb8f919d700 1 -- 172.21.15.6:0/163508396 wait complete. 2023-11-17T00:08:15.837 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:08:15.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3203428558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/721822387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4145327208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3203428558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/721822387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:15.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4145327208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:16.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 -- 172.21.15.6:0/3433458393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 msgr2=0x7f33f81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 --2- 172.21.15.6:0/3433458393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81006e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009990 tx=0x7f33f4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 -- 172.21.15.6:0/3433458393 shutdown_connections 2023-11-17T00:08:16.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 --2- 172.21.15.6:0/3433458393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81006e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 --2- 172.21.15.6:0/3433458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 --2- 172.21.15.6:0/3433458393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 0x7f33f8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 -- 172.21.15.6:0/3433458393 >> 172.21.15.6:0/3433458393 conn(0x7f33f80fb8b0 msgr2=0x7f33f80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:16.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 -- 172.21.15.6:0/3433458393 shutdown_connections 2023-11-17T00:08:16.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.040+0000 7f33ffdff700 1 -- 172.21.15.6:0/3433458393 wait complete. 2023-11-17T00:08:16.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 Processor -- start 2023-11-17T00:08:16.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 -- start start 2023-11-17T00:08:16.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 0x7f33f8196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819dc60 con 0x7f33f81002d0 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819dda0 con 0x7f33f8101e50 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.041+0000 7f33ffdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819df30 con 0x7f33f81014c0 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59772/0 (socket says 172.21.15.6:59772) 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 -- 172.21.15.6:0/2044245924 learned_addr learned my addr 172.21.15.6:0/2044245924 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fd39a700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 0x7f33f8196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fe39c700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 -- 172.21.15.6:0/2044245924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 msgr2=0x7f33f8196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 0x7f33f8196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 -- 172.21.15.6:0/2044245924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 msgr2=0x7f33f819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 -- 172.21.15.6:0/2044245924 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0009c40 con 0x7f33f81002d0 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fe39c700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.042+0000 7f33fdb9b700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33f400b620 tx=0x7f33f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.043+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f401d070 con 0x7f33f81002d0 2023-11-17T00:08:16.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.043+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f4005030 con 0x7f33f81002d0 2023-11-17T00:08:16.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.043+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f40095f0 con 0x7f33f81002d0 2023-11-17T00:08:16.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.043+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4004bf0 con 0x7f33f81002d0 2023-11-17T00:08:16.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.043+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f810bc30 con 0x7f33f81002d0 2023-11-17T00:08:16.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.044+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f8107860 con 0x7f33f81002d0 2023-11-17T00:08:16.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.047+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f33f4004d50 con 0x7f33f81002d0 2023-11-17T00:08:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.049+0000 7f33eeffd700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 0x7f33e40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.049+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f33f40cdc70 con 0x7f33f81002d0 2023-11-17T00:08:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.049+0000 7f33fd39a700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 0x7f33e40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.049+0000 7f33fd39a700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 0x7f33e40a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009c60 tx=0x7f33e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:16.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.053+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f33f4097080 con 0x7f33f81002d0 2023-11-17T00:08:16.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.308+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33f802cfe0 con 0x7f33f81002d0 2023-11-17T00:08:16.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.309+0000 7f33eeffd700 1 -- 172.21.15.6:0/2044245924 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f33f409a6a0 con 0x7f33f81002d0 2023-11-17T00:08:16.310 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:16.311 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 msgr2=0x7f33e40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 0x7f33e40a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009c60 tx=0x7f33e8009260 comp rx=0 tx=0).stop 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 msgr2=0x7f33f81960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33f400b620 tx=0x7f33f4004730 comp rx=0 tx=0).stop 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 shutdown_connections 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f33e40a11d0 0x7f33e40a3680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33f81002d0 0x7f33f81960e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33f8101e50 0x7f33f819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 --2- 172.21.15.6:0/2044245924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33f81014c0 0x7f33f8196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 >> 172.21.15.6:0/2044245924 conn(0x7f33f80fb8b0 msgr2=0x7f33f81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 shutdown_connections 2023-11-17T00:08:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.311+0000 7f33ffdff700 1 -- 172.21.15.6:0/2044245924 wait complete. 2023-11-17T00:08:16.314 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:08:16.332 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 -- 172.21.15.6:0/2515797724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14101002d0 msgr2=0x7f14101006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 --2- 172.21.15.6:0/2515797724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14101002d0 0x7f14101006e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1400009990 tx=0x7f1400009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 -- 172.21.15.6:0/2515797724 shutdown_connections 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 --2- 172.21.15.6:0/2515797724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14101002d0 0x7f14101006e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 --2- 172.21.15.6:0/2515797724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1410101e50 0x7f1410108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 --2- 172.21.15.6:0/2515797724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f1410101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 -- 172.21.15.6:0/2515797724 >> 172.21.15.6:0/2515797724 conn(0x7f14100fb8b0 msgr2=0x7f14100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 -- 172.21.15.6:0/2515797724 shutdown_connections 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.539+0000 7f14163ff700 1 -- 172.21.15.6:0/2515797724 wait complete. 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 Processor -- start 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 -- start start 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f14101987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1410101e50 0x7f141019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141019fe10 con 0x7f1410101e50 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141019ff50 con 0x7f14101014c0 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.540+0000 7f14163ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14101a00e0 con 0x7f14101002d0 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:50838/0 (socket says 172.21.15.6:50838) 2023-11-17T00:08:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 -- 172.21.15.6:0/3288471822 learned_addr learned my addr 172.21.15.6:0/3288471822 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140f7fe700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f14101987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f141499c700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1410101e50 0x7f141019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 -- 172.21.15.6:0/3288471822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 msgr2=0x7f14101987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f14101987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 -- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1410101e50 msgr2=0x7f141019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1410101e50 0x7f141019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 -- 172.21.15.6:0/3288471822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000095f0 con 0x7f14101002d0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140f7fe700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f14101987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.541+0000 7f140ffff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1400004730 tx=0x7f1400004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.542+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140001d070 con 0x7f14101002d0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.542+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1400021410 con 0x7f14101002d0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.542+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141010bca0 con 0x7f14101002d0 2023-11-17T00:08:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.542+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400017650 con 0x7f14101002d0 2023-11-17T00:08:16.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.542+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141010bb10 con 0x7f14101002d0 2023-11-17T00:08:16.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.543+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1410192a00 con 0x7f14101002d0 2023-11-17T00:08:16.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.545+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1400021580 con 0x7f14101002d0 2023-11-17T00:08:16.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.546+0000 7f140d7fa700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 0x7f13f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:16.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.546+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f14000ce780 con 0x7f14101002d0 2023-11-17T00:08:16.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.547+0000 7f140f7fe700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 0x7f13f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:16.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.547+0000 7f140f7fe700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 0x7f13f80a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1404011490 tx=0x7f1404009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.548+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1400097b90 con 0x7f14101002d0 2023-11-17T00:08:16.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:16 smithi006 ceph-mon[97840]: pgmap v359: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 858 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/163508396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:16.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2044245924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.758+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f141002cfe0 con 0x7f14101002d0 2023-11-17T00:08:16.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.760+0000 7f140d7fa700 1 -- 172.21.15.6:0/3288471822 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7f140009b1b0 con 0x7f14101002d0 2023-11-17T00:08:16.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:16.761 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:16.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 msgr2=0x7f13f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 0x7f13f80a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1404011490 tx=0x7f1404009260 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 msgr2=0x7f1410198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1400004730 tx=0x7f1400004760 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 shutdown_connections 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f13f80a1150 0x7f13f80a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1410101e50 0x7f141019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14101002d0 0x7f1410198290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 --2- 172.21.15.6:0/3288471822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14101014c0 0x7f14101987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 >> 172.21.15.6:0/3288471822 conn(0x7f14100fb8b0 msgr2=0x7f1410106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:16.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 shutdown_connections 2023-11-17T00:08:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.763+0000 7f14163ff700 1 -- 172.21.15.6:0/3288471822 wait complete. 2023-11-17T00:08:16.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:08:16.783 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:08:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:16 smithi087 ceph-mon[131182]: pgmap v359: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 858 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/163508396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2044245924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:16 smithi186 ceph-mon[131487]: pgmap v359: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 858 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/163508396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2044245924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 -- 172.21.15.6:0/7220371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0101e10 msgr2=0x7f52e01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 --2- 172.21.15.6:0/7220371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0101e10 0x7f52e01086d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009f90 tx=0x7f52d800b330 comp rx=0 tx=0).stop 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 -- 172.21.15.6:0/7220371 shutdown_connections 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 --2- 172.21.15.6:0/7220371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0101e10 0x7f52e01086d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 --2- 172.21.15.6:0/7220371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0100290 0x7f52e01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 --2- 172.21.15.6:0/7220371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 -- 172.21.15.6:0/7220371 >> 172.21.15.6:0/7220371 conn(0x7f52e00fb870 msgr2=0x7f52e00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:16.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 -- 172.21.15.6:0/7220371 shutdown_connections 2023-11-17T00:08:17.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.991+0000 7f52e5fff700 1 -- 172.21.15.6:0/7220371 wait complete. 2023-11-17T00:08:17.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.992+0000 7f52e5fff700 1 Processor -- start 2023-11-17T00:08:17.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.992+0000 7f52e5fff700 1 -- start start 2023-11-17T00:08:17.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.992+0000 7f52e5fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52e5fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52e5fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101e10 0x7f52e019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52e5fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e019fe70 con 0x7f52e0100290 2023-11-17T00:08:17.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52e5fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e019ffb0 con 0x7f52e0101480 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52e5fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01a0140 con 0x7f52e0101e10 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59802/0 (socket says 172.21.15.6:59802) 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 -- 172.21.15.6:0/4199135356 learned_addr learned my addr 172.21.15.6:0/4199135356 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52616/0 (socket says 172.21.15.6:52616) 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52dffff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101e10 0x7f52e019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 -- 172.21.15.6:0/4199135356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101e10 msgr2=0x7f52e019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101e10 0x7f52e019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 -- 172.21.15.6:0/4199135356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 msgr2=0x7f52e0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.993+0000 7f52df7fe700 1 -- 172.21.15.6:0/4199135356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80095f0 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52df7fe700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e970 tx=0x7f52d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0009790 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d000c7e0 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d8009c40 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003c70 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.994+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e010bc70 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.995+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e01928e0 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.997+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f52d000cc50 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.998+0000 7f52dd7fa700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 0x7f52cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.998+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f52d0014070 con 0x7f52e0100290 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.999+0000 7f52d7fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 0x7f52cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:16.999+0000 7f52d7fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 0x7f52cc0a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.002+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f52d00d3080 con 0x7f52e0100290 2023-11-17T00:08:17.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.216+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52e0060f30 con 0x7f52e0100290 2023-11-17T00:08:17.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.217+0000 7f52dd7fa700 1 -- 172.21.15.6:0/4199135356 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52d0096200 con 0x7f52e0100290 2023-11-17T00:08:17.217 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:08:17.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.219+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 msgr2=0x7f52cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.219+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 0x7f52cc0a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8009310 comp rx=0 tx=0).stop 2023-11-17T00:08:17.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.219+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 msgr2=0x7f52e01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.219+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e970 tx=0x7f52d000ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:17.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 shutdown_connections 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52cc0a1150 0x7f52cc0a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52e0100290 0x7f52e01982f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52e0101e10 0x7f52e019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 --2- 172.21.15.6:0/4199135356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52e0101480 0x7f52e0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 >> 172.21.15.6:0/4199135356 conn(0x7f52e00fb870 msgr2=0x7f52e0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 shutdown_connections 2023-11-17T00:08:17.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.220+0000 7f52e5fff700 1 -- 172.21.15.6:0/4199135356 wait complete. 2023-11-17T00:08:17.240 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.449+0000 7f9c60dff700 1 -- 172.21.15.6:0/545285292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 msgr2=0x7f9c5c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.449+0000 7f9c60dff700 1 --2- 172.21.15.6:0/545285292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c1006a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 -- 172.21.15.6:0/545285292 shutdown_connections 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 --2- 172.21.15.6:0/545285292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c1006a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 --2- 172.21.15.6:0/545285292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101e10 0x7f9c5c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 --2- 172.21.15.6:0/545285292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101480 0x7f9c5c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 -- 172.21.15.6:0/545285292 >> 172.21.15.6:0/545285292 conn(0x7f9c5c0fb870 msgr2=0x7f9c5c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 -- 172.21.15.6:0/545285292 shutdown_connections 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.450+0000 7f9c60dff700 1 -- 172.21.15.6:0/545285292 wait complete. 2023-11-17T00:08:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 Processor -- start 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 -- start start 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101480 0x7f9c5c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 0x7f9c5c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c073bc0 con 0x7f9c5c100290 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c073d30 con 0x7f9c5c101e10 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c60dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1a0060 con 0x7f9c5c101480 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c5a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c5a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59820/0 (socket says 172.21.15.6:59820) 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c59d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101480 0x7f9c5c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.451+0000 7f9c5a59c700 1 -- 172.21.15.6:0/4209565506 learned_addr learned my addr 172.21.15.6:0/4209565506 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5ad9d700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 0x7f9c5c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 -- 172.21.15.6:0/4209565506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101480 msgr2=0x7f9c5c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101480 0x7f9c5c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 -- 172.21.15.6:0/4209565506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 msgr2=0x7f9c5c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 0x7f9c5c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 -- 172.21.15.6:0/4209565506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5ad9d700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 0x7f9c5c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c5a59c700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005740 tx=0x7f9c4c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.452+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d070 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.453+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4c021410 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.453+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c10bc40 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.453+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c017650 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.453+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c10bab0 con 0x7f9c5c100290 2023-11-17T00:08:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.454+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c067a50 con 0x7f9c5c100290 2023-11-17T00:08:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.456+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9c4c021580 con 0x7f9c5c100290 2023-11-17T00:08:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.458+0000 7f9c4b7fe700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 0x7f9c440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.458+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9c4c0ce1e0 con 0x7f9c5c100290 2023-11-17T00:08:17.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.458+0000 7f9c59d9b700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 0x7f9c440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.459+0000 7f9c59d9b700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 0x7f9c440a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009c40 tx=0x7f9c50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.459+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9c4c0975f0 con 0x7f9c5c100290 2023-11-17T00:08:17.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.670+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c5c107200 con 0x7f9c5c100290 2023-11-17T00:08:17.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.671+0000 7f9c4b7fe700 1 -- 172.21.15.6:0/4209565506 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c4c09ac10 con 0x7f9c5c100290 2023-11-17T00:08:17.672 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:17.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 msgr2=0x7f9c440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 0x7f9c440a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009c40 tx=0x7f9c50009260 comp rx=0 tx=0).stop 2023-11-17T00:08:17.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 msgr2=0x7f9c5c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005740 tx=0x7f9c4c005770 comp rx=0 tx=0).stop 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 shutdown_connections 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f9c440a11d0 0x7f9c440a3680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c5c100290 0x7f9c5c0748a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c5c101e10 0x7f9c5c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 --2- 172.21.15.6:0/4209565506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c5c101480 0x7f9c5c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 >> 172.21.15.6:0/4209565506 conn(0x7f9c5c0fb870 msgr2=0x7f9c5c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 shutdown_connections 2023-11-17T00:08:17.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.673+0000 7f9c60dff700 1 -- 172.21.15.6:0/4209565506 wait complete. 2023-11-17T00:08:17.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3288471822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:17.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4199135356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:17.693 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:08:17.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3288471822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:17.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4199135356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:17.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3288471822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:17.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4199135356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 -- 172.21.15.6:0/1932690059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 msgr2=0x7f1d2c1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 --2- 172.21.15.6:0/1932690059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c1086d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400b0a0 tx=0x7f1d2400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 -- 172.21.15.6:0/1932690059 shutdown_connections 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 --2- 172.21.15.6:0/1932690059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c1086d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 --2- 172.21.15.6:0/1932690059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 0x7f1d2c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 --2- 172.21.15.6:0/1932690059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 0x7f1d2c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 -- 172.21.15.6:0/1932690059 >> 172.21.15.6:0/1932690059 conn(0x7f1d2c0fb870 msgr2=0x7f1d2c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 -- 172.21.15.6:0/1932690059 shutdown_connections 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.907+0000 7f1d309ff700 1 -- 172.21.15.6:0/1932690059 wait complete. 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 Processor -- start 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 -- start start 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 0x7f1d2c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 0x7f1d2c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.909+0000 7f1d309ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c19fe70 con 0x7f1d2c101e10 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d309ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c19ffb0 con 0x7f1d2c101480 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d309ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c1a0140 con 0x7f1d2c100290 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d29d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 0x7f1d2c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 0x7f1d2c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59836/0 (socket says 172.21.15.6:59836) 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 -- 172.21.15.6:0/2529262338 learned_addr learned my addr 172.21.15.6:0/2529262338 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 -- 172.21.15.6:0/2529262338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 msgr2=0x7f1d2c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 0x7f1d2c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 -- 172.21.15.6:0/2529262338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 msgr2=0x7f1d2c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 0x7f1d2c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 -- 172.21.15.6:0/2529262338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d24009d00 con 0x7f1d2c101e10 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.910+0000 7f1d2ad9d700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400d010 tx=0x7f1d2400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.911+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400e040 con 0x7f1d2c101e10 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.911+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d24003e10 con 0x7f1d2c101e10 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.911+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d240052a0 con 0x7f1d2c101e10 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.911+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c10bc70 con 0x7f1d2c101e10 2023-11-17T00:08:17.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.911+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c10bae0 con 0x7f1d2c101e10 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.912+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c1928e0 con 0x7f1d2c101e10 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.914+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1d2401d480 con 0x7f1d2c101e10 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.915+0000 7f1d1b7fe700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 0x7f1d140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.915+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1d240cdfb0 con 0x7f1d2c101e10 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.915+0000 7f1d2a59c700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 0x7f1d140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.916+0000 7f1d2a59c700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 0x7f1d140a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c1012e0 tx=0x7f1d1c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:17.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:17.919+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1d240973c0 con 0x7f1d2c101e10 2023-11-17T00:08:18.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.131+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d2c061cc0 con 0x7f1d2c101e10 2023-11-17T00:08:18.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.131+0000 7f1d1b7fe700 1 -- 172.21.15.6:0/2529262338 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d2409a9e0 con 0x7f1d2c101e10 2023-11-17T00:08:18.132 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:08:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.133+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 msgr2=0x7f1d140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 0x7f1d140a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c1012e0 tx=0x7f1d1c009260 comp rx=0 tx=0).stop 2023-11-17T00:08:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 msgr2=0x7f1d2c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400d010 tx=0x7f1d2400bc80 comp rx=0 tx=0).stop 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 shutdown_connections 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1d140a11d0 0x7f1d140a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1d2c101e10 0x7f1d2c19f8a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c100290 0x7f1d2c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 --2- 172.21.15.6:0/2529262338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d2c101480 0x7f1d2c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 >> 172.21.15.6:0/2529262338 conn(0x7f1d2c0fb870 msgr2=0x7f1d2c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:18.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 shutdown_connections 2023-11-17T00:08:18.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.134+0000 7f1d309ff700 1 -- 172.21.15.6:0/2529262338 wait complete. 2023-11-17T00:08:18.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 -- 172.21.15.6:0/1585106813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384100290 msgr2=0x7f33841006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 --2- 172.21.15.6:0/1585106813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384100290 0x7f33841006a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3374009990 tx=0x7f3374009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 -- 172.21.15.6:0/1585106813 shutdown_connections 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 --2- 172.21.15.6:0/1585106813 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384101e10 0x7f33841086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 --2- 172.21.15.6:0/1585106813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384100290 0x7f33841006a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 --2- 172.21.15.6:0/1585106813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101480 0x7f33841018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 -- 172.21.15.6:0/1585106813 >> 172.21.15.6:0/1585106813 conn(0x7f33840fb870 msgr2=0x7f33840fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 -- 172.21.15.6:0/1585106813 shutdown_connections 2023-11-17T00:08:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.367+0000 7f3389fff700 1 -- 172.21.15.6:0/1585106813 wait complete. 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.368+0000 7f3389fff700 1 Processor -- start 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.368+0000 7f3389fff700 1 -- start start 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384101480 0x7f33841964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101e10 0x7f338419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419db00 con 0x7f3384100290 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419dc40 con 0x7f3384101e10 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3389fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338419ddd0 con 0x7f3384101480 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59858/0 (socket says 172.21.15.6:59858) 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 -- 172.21.15.6:0/3809011859 learned_addr learned my addr 172.21.15.6:0/3809011859 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3382ffd700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384101480 0x7f33841964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f3383fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101e10 0x7f338419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 -- 172.21.15.6:0/3809011859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384101480 msgr2=0x7f33841964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384101480 0x7f33841964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 -- 172.21.15.6:0/3809011859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101e10 msgr2=0x7f338419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101e10 0x7f338419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.369+0000 7f33837fe700 1 -- 172.21.15.6:0/3809011859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740095f0 con 0x7f3384100290 2023-11-17T00:08:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f33837fe700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3374004710 tx=0x7f3374009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337401d070 con 0x7f3384100290 2023-11-17T00:08:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338410bbe0 con 0x7f3384100290 2023-11-17T00:08:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3374021410 con 0x7f3384100290 2023-11-17T00:08:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374017650 con 0x7f3384100290 2023-11-17T00:08:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.370+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338410ba50 con 0x7f3384100290 2023-11-17T00:08:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.371+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338419e3d0 con 0x7f3384100290 2023-11-17T00:08:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.373+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3374021580 con 0x7f3384100290 2023-11-17T00:08:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.374+0000 7f3380ff9700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 0x7f336c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.375+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f33740ce070 con 0x7f3384100290 2023-11-17T00:08:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.375+0000 7f3382ffd700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 0x7f336c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.375+0000 7f3382ffd700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 0x7f336c0a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3378009c60 tx=0x7f3378009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:18.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.378+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3374097480 con 0x7f3384100290 2023-11-17T00:08:18.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.597+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3384107230 con 0x7f3384100290 2023-11-17T00:08:18.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.598+0000 7f3380ff9700 1 -- 172.21.15.6:0/3809011859 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f337409aaa0 con 0x7f3384100290 2023-11-17T00:08:18.598 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 msgr2=0x7f336c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 0x7f336c0a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3378009c60 tx=0x7f3378009260 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 msgr2=0x7f3384195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3374004710 tx=0x7f3374009580 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 shutdown_connections 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f336c0a1150 0x7f336c0a3600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3384100290 0x7f3384195f80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3384101e10 0x7f338419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 --2- 172.21.15.6:0/3809011859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3384101480 0x7f33841964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.600+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 >> 172.21.15.6:0/3809011859 conn(0x7f33840fb870 msgr2=0x7f33840fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.601+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 shutdown_connections 2023-11-17T00:08:18.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.601+0000 7f3389fff700 1 -- 172.21.15.6:0/3809011859 wait complete. 2023-11-17T00:08:18.621 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:08:18.686 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4209565506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:18.686 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:18 smithi006 ceph-mon[97840]: pgmap v360: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:18.687 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2529262338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 -- 172.21.15.6:0/69897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 msgr2=0x7f526c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 --2- 172.21.15.6:0/69897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c1006e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5268009990 tx=0x7f5268009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 -- 172.21.15.6:0/69897227 shutdown_connections 2023-11-17T00:08:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 --2- 172.21.15.6:0/69897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c1006e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 --2- 172.21.15.6:0/69897227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c101e50 0x7f526c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 --2- 172.21.15.6:0/69897227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c1014c0 0x7f526c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 -- 172.21.15.6:0/69897227 >> 172.21.15.6:0/69897227 conn(0x7f526c0fb8b0 msgr2=0x7f526c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:18.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 -- 172.21.15.6:0/69897227 shutdown_connections 2023-11-17T00:08:18.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.830+0000 7f52733ff700 1 -- 172.21.15.6:0/69897227 wait complete. 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.831+0000 7f52733ff700 1 Processor -- start 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.831+0000 7f52733ff700 1 -- start start 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.831+0000 7f52733ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.831+0000 7f52733ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 0x7f526c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.831+0000 7f52733ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c101e50 0x7f526c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f52733ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19fe10 con 0x7f526c1002d0 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f52733ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19ff50 con 0x7f526c1014c0 2023-11-17T00:08:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f52733ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c1a00e0 con 0x7f526c101e50 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34460/0 (socket says 172.21.15.6:34460) 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 -- 172.21.15.6:0/2564616127 learned_addr learned my addr 172.21.15.6:0/2564616127 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527099a700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 0x7f526c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527199c700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c101e50 0x7f526c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 -- 172.21.15.6:0/2564616127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c101e50 msgr2=0x7f526c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c101e50 0x7f526c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 -- 172.21.15.6:0/2564616127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 msgr2=0x7f526c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 0x7f526c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 -- 172.21.15.6:0/2564616127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680095f0 con 0x7f526c1002d0 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527099a700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 0x7f526c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.832+0000 7f527119b700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5268004730 tx=0x7f5268004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.833+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526801d070 con 0x7f526c1002d0 2023-11-17T00:08:18.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.833+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c10bca0 con 0x7f526c1002d0 2023-11-17T00:08:18.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.833+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5268021410 con 0x7f526c1002d0 2023-11-17T00:08:18.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.833+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c10bb10 con 0x7f526c1002d0 2023-11-17T00:08:18.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.833+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680175f0 con 0x7f526c1002d0 2023-11-17T00:08:18.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.834+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c192a00 con 0x7f526c1002d0 2023-11-17T00:08:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.836+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5268021580 con 0x7f526c1002d0 2023-11-17T00:08:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.837+0000 7f52627fc700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 0x7f52580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.837+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f52680cde30 con 0x7f526c1002d0 2023-11-17T00:08:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.837+0000 7f527099a700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 0x7f52580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.838+0000 7f527099a700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 0x7f52580a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f525c011490 tx=0x7f525c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:18.841+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5268097360 con 0x7f526c1002d0 2023-11-17T00:08:18.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4209565506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:18 smithi087 ceph-mon[131182]: pgmap v360: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2529262338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:18.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4209565506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:18 smithi186 ceph-mon[131487]: pgmap v360: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2023-11-17T00:08:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2529262338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:19.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.100+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f526c02cfe0 con 0x7f526c1002d0 2023-11-17T00:08:19.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.101+0000 7f52627fc700 1 -- 172.21.15.6:0/2564616127 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f526809a980 con 0x7f526c1002d0 2023-11-17T00:08:19.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:19.101 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 msgr2=0x7f52580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 0x7f52580a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f525c011490 tx=0x7f525c009260 comp rx=0 tx=0).stop 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 msgr2=0x7f526c198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5268004730 tx=0x7f5268004760 comp rx=0 tx=0).stop 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 shutdown_connections 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f52580a1150 0x7f52580a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f526c1002d0 0x7f526c198290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f526c101e50 0x7f526c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 --2- 172.21.15.6:0/2564616127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f526c1014c0 0x7f526c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 >> 172.21.15.6:0/2564616127 conn(0x7f526c0fb8b0 msgr2=0x7f526c106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 shutdown_connections 2023-11-17T00:08:19.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.103+0000 7f52733ff700 1 -- 172.21.15.6:0/2564616127 wait complete. 2023-11-17T00:08:19.105 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:08:19.123 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:19.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.326+0000 7f550e1ff700 1 -- 172.21.15.6:0/3542741768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55081014c0 msgr2=0x7f5508101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.326+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3542741768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55081014c0 0x7f5508101910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009990 tx=0x7f54fc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.326+0000 7f550e1ff700 1 -- 172.21.15.6:0/3542741768 shutdown_connections 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3542741768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5508101e50 0x7f5508108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3542741768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f55081006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3542741768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55081014c0 0x7f5508101910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 -- 172.21.15.6:0/3542741768 >> 172.21.15.6:0/3542741768 conn(0x7f55080fb8b0 msgr2=0x7f55080fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 -- 172.21.15.6:0/3542741768 shutdown_connections 2023-11-17T00:08:19.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.327+0000 7f550e1ff700 1 -- 172.21.15.6:0/3542741768 wait complete. 2023-11-17T00:08:19.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.328+0000 7f550e1ff700 1 Processor -- start 2023-11-17T00:08:19.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.328+0000 7f550e1ff700 1 -- start start 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55081014c0 0x7f5508196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508101e50 0x7f550819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819db90 con 0x7f55081014c0 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819dcd0 con 0x7f55081002d0 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f550e1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819de60 con 0x7f5508101e50 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55081014c0 0x7f5508196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43966/0 (socket says 172.21.15.6:43966) 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 -- 172.21.15.6:0/3858224292 learned_addr learned my addr 172.21.15.6:0/3858224292 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f5507fff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508101e50 0x7f550819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 -- 172.21.15.6:0/3858224292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508101e50 msgr2=0x7f550819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508101e50 0x7f550819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 -- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55081014c0 msgr2=0x7f5508196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55081014c0 0x7f5508196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.329+0000 7f55077fe700 1 -- 172.21.15.6:0/3858224292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0095f0 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f55077fe700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e970 tx=0x7f54f800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8009790 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550810bc30 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f800c7e0 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003c70 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.330+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550810baa0 con 0x7f55081002d0 2023-11-17T00:08:19.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.331+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5508107870 con 0x7f55081002d0 2023-11-17T00:08:19.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.333+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f54f800cc50 con 0x7f55081002d0 2023-11-17T00:08:19.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.335+0000 7f5504ff9700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 0x7f54f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.335+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f54f8014070 con 0x7f55081002d0 2023-11-17T00:08:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.335+0000 7f5506ffd700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 0x7f54f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.335+0000 7f5506ffd700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 0x7f54f00a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b2e0 tx=0x7f54fc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.337+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f54f8096350 con 0x7f55081002d0 2023-11-17T00:08:19.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.549+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f550802cfe0 con 0x7f55081002d0 2023-11-17T00:08:19.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.551+0000 7f5504ff9700 1 -- 172.21.15.6:0/3858224292 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7f54f8003dd0 con 0x7f55081002d0 2023-11-17T00:08:19.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:19.553 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:19.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 msgr2=0x7f54f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 0x7f54f00a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b2e0 tx=0x7f54fc005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 msgr2=0x7f5508196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e970 tx=0x7f54f800ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 shutdown_connections 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f54f00a1150 0x7f54f00a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508101e50 0x7f550819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081002d0 0x7f5508196010 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 --2- 172.21.15.6:0/3858224292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f55081014c0 0x7f5508196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 >> 172.21.15.6:0/3858224292 conn(0x7f55080fb8b0 msgr2=0x7f55081049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 shutdown_connections 2023-11-17T00:08:19.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.554+0000 7f550e1ff700 1 -- 172.21.15.6:0/3858224292 wait complete. 2023-11-17T00:08:19.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-17T00:06:12.598183+0000', 'modified': '2023-11-17T00:06:37.833900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6837/850260336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 850260336}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 850260336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.87:6839/804288298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 804288298}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 804288298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-17T00:08:19.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-17T00:08:19.576 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-17T00:08:19.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3809011859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:19.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2564616127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.780+0000 7f81579ff700 1 -- 172.21.15.6:0/2186547529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150101dc0 msgr2=0x7f8150108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.780+0000 7f81579ff700 1 --2- 172.21.15.6:0/2186547529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150101dc0 0x7f8150108680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f814800b0a0 tx=0x7f814800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 -- 172.21.15.6:0/2186547529 shutdown_connections 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 --2- 172.21.15.6:0/2186547529 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150101dc0 0x7f8150108680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 --2- 172.21.15.6:0/2186547529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150100240 0x7f8150100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 --2- 172.21.15.6:0/2186547529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 0x7f8150101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 -- 172.21.15.6:0/2186547529 >> 172.21.15.6:0/2186547529 conn(0x7f81500fb820 msgr2=0x7f81500fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 -- 172.21.15.6:0/2186547529 shutdown_connections 2023-11-17T00:08:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.781+0000 7f81579ff700 1 -- 172.21.15.6:0/2186547529 wait complete. 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 Processor -- start 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 -- start start 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 0x7f8150198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150101dc0 0x7f815019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019fe70 con 0x7f8150100240 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019ffb0 con 0x7f8150101430 2023-11-17T00:08:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.782+0000 7f81579ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81501a0140 con 0x7f8150101dc0 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34484/0 (socket says 172.21.15.6:34484) 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 -- 172.21.15.6:0/1360620117 learned_addr learned my addr 172.21.15.6:0/1360620117 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 0x7f8150198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f8155f9c700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150101dc0 0x7f815019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 -- 172.21.15.6:0/1360620117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150101dc0 msgr2=0x7f815019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150101dc0 0x7f815019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 -- 172.21.15.6:0/1360620117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 msgr2=0x7f8150198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 0x7f8150198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 -- 172.21.15.6:0/1360620117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148009d00 con 0x7f8150100240 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f815579b700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d980 tx=0x7f814c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f81467fc700 1 -- 172.21.15.6:0/1360620117 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00f890 con 0x7f8150100240 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.783+0000 7f81467fc700 1 -- 172.21.15.6:0/1360620117 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c010460 con 0x7f8150100240 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.784+0000 7f81467fc700 1 -- 172.21.15.6:0/1360620117 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0195c0 con 0x7f8150100240 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.784+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815010bc20 con 0x7f8150100240 2023-11-17T00:08:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.784+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815010ba90 con 0x7f8150100240 2023-11-17T00:08:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.787+0000 7f81467fc700 1 -- 172.21.15.6:0/1360620117 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f814c0105d0 con 0x7f8150100240 2023-11-17T00:08:19.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 Processor -- start 2023-11-17T00:08:19.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 -- start start 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f81467fc700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 0x7f813c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f81467fc700 1 -- 172.21.15.6:0/1360620117 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f814c0cc9c0 con 0x7f8150100240 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601880db010 con 0x5601881d5bb0 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018812d850 con 0x5601881d70c0 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f816b422b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560188126ce0 con 0x5601881d6500 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.788+0000 7f8155f9c700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 0x7f813c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34492/0 (socket says 172.21.15.6:34492) 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 -- 172.21.15.6:0/645632538 learned_addr learned my addr 172.21.15.6:0/645632538 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f8154f9a700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f8155f9c700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 -- 172.21.15.6:0/645632538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 msgr2=0x5601881d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881d6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 -- 172.21.15.6:0/645632538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 msgr2=0x5601881d7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881d7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 -- 172.21.15.6:0/645632538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560187fa85e0 con 0x5601881d5bb0 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f8155f9c700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f8155f9c700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 0x7f813c0a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f814800b830 tx=0x7f8148007740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.789+0000 7f815579b700 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f814c099990 tx=0x7f814c099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848ea68fd079e77e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f813b7fe700 1 -- 172.21.15.6:0/645632538 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c09da30 con 0x5601881d5bb0 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f813b7fe700 1 -- 172.21.15.6:0/645632538 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c0a1e50 con 0x5601881d5bb0 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f813b7fe700 1 -- 172.21.15.6:0/645632538 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0a19c0 con 0x5601881d5bb0 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 -- 172.21.15.6:0/645632538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 msgr2=0x5601881d5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f814c099990 tx=0x7f814c099430 comp rx=0 tx=0).stop 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 -- 172.21.15.6:0/645632538 shutdown_connections 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881d5fc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 --2- 172.21.15.6:0/645632538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 -- 172.21.15.6:0/645632538 >> 172.21.15.6:0/645632538 conn(0x5601881cb310 msgr2=0x5601881d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 -- 172.21.15.6:0/645632538 shutdown_connections 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.790+0000 7f816b422b80 1 -- 172.21.15.6:0/645632538 wait complete. 2023-11-17T00:08:19.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.791+0000 7f816b422b80 1 -- start start 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.791+0000 7f816b422b80 1 Processor -- start 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d6500 0x5601881df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601880c8890 con 0x5601881d6500 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601880c8530 con 0x5601881d70c0 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560187faa8d0 con 0x5601881d5bb0 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f816b422b80 10 client.0 _ll_register_callbacks cb 0x5601880e6730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59858/0 (socket says 172.21.15.6:59858) 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 -- 172.21.15.6:0/1204753790 learned_addr learned my addr 172.21.15.6:0/1204753790 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d6500 0x5601881df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f8155f9c700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 -- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 msgr2=0x5601881e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 -- 172.21.15.6:0/1204753790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d6500 msgr2=0x5601881df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d6500 0x5601881df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.792+0000 7f815579b700 1 -- 172.21.15.6:0/1204753790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815010ba90 con 0x5601881d5bb0 2023-11-17T00:08:19.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f815579b700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f814c007e60 tx=0x7f814c09da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e07a4e5b7ca108 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f8138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c097da0 con 0x5601881d5bb0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601881c8ad0 con 0x5601881d5bb0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f814c0a19c0 con 0x5601881d5bb0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f814c0a1bc0 con 0x5601881d5bb0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f816b422b80 10 client.25252 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.793+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5601880ed140 con 0x5601881d5bb0 2023-11-17T00:08:19.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601881c8ad0 con 0x5601881d5bb0 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f814c0b91d0 con 0x5601881d5bb0 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 10 client.25252 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 10 client.25252 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 20 client.25252 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5601881ea7e0 0x5601881e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 4 client.25252 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601880e9050 con 0x5601881ea7e0 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.794+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5601881ea7e0 0x5601881e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.795+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5601881ea7e0 0x5601881e9870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9b8a89dcdb08e3 server_cookie=b4cf48ef56406a63 in_seq=0 out_seq=0 2023-11-17T00:08:19.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.795+0000 7f8138ff9700 10 client.25252 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.795+0000 7f8138ff9700 1 -- 172.21.15.6:0/1204753790 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5601880e9050 con 0x5601881ea7e0 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f8138ff9700 10 client.25252 handle_command_reply: tid=0 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 client.25252 shutdown 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 msgr2=0x5601881df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f814c007e60 tx=0x7f814c09da30 comp rx=0 tx=0).stop 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 shutdown_connections 2023-11-17T00:08:19.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d5bb0 0x5601881df450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5601881ea7e0 0x5601881e9870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 --2- 172.21.15.6:0/1204753790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d6500 0x5601881df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f8138ff9700 0 client.25252 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 >> 172.21.15.6:0/1204753790 conn(0x5601881cb310 msgr2=0x5601881cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 shutdown_connections 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 1 -- 172.21.15.6:0/1204753790 wait complete. 2023-11-17T00:08:19.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.796+0000 7f816b422b80 20 client.25252 trim_cache size 0 max 16384 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.814+0000 7f816b422b80 1 Processor -- start 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 -- start start 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601880db010 con 0x5601881d5bb0 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601881d68d0 con 0x5601881d70c0 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f816b422b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601880e9050 con 0x5601881d6500 2023-11-17T00:08:19.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34504/0 (socket says 172.21.15.6:34504) 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f815579b700 1 -- 172.21.15.6:0/1939246625 learned_addr learned my addr 172.21.15.6:0/1939246625 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f8155f9c700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f815579b700 1 -- 172.21.15.6:0/1939246625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 msgr2=0x5601881e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.815+0000 7f815579b700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f815579b700 1 -- 172.21.15.6:0/1939246625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 msgr2=0x5601881dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f815579b700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f815579b700 1 -- 172.21.15.6:0/1939246625 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601880ece90 con 0x5601881d5bb0 2023-11-17T00:08:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f8154f9a700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f815579b700 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f814c099990 tx=0x7f814c099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6508000a4c56d0c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f813bfff700 1 -- 172.21.15.6:0/1939246625 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c09daa0 con 0x5601881d5bb0 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f813bfff700 1 -- 172.21.15.6:0/1939246625 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c084410 con 0x5601881d5bb0 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.816+0000 7f813bfff700 1 -- 172.21.15.6:0/1939246625 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c084600 con 0x5601881d5bb0 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- 172.21.15.6:0/1939246625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 msgr2=0x5601881e9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f814c099990 tx=0x7f814c099430 comp rx=0 tx=0).stop 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- 172.21.15.6:0/1939246625 shutdown_connections 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e9870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x5601881dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 --2- 172.21.15.6:0/1939246625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- 172.21.15.6:0/1939246625 >> 172.21.15.6:0/1939246625 conn(0x5601881cb310 msgr2=0x5601881de0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- 172.21.15.6:0/1939246625 shutdown_connections 2023-11-17T00:08:19.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- 172.21.15.6:0/1939246625 wait complete. 2023-11-17T00:08:19.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 -- start start 2023-11-17T00:08:19.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.817+0000 7f816b422b80 1 Processor -- start 2023-11-17T00:08:19.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x56018823e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601881e6b60 con 0x5601881d5bb0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601881e6cd0 con 0x5601881d70c0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018823e590 con 0x5601881d6500 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f816b422b80 10 client.0 _ll_register_callbacks cb 0x7f814000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34512/0 (socket says 172.21.15.6:34512) 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f8155f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x56018823e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 -- 172.21.15.6:0/784097491 learned_addr learned my addr 172.21.15.6:0/784097491 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f8154f9a700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 -- 172.21.15.6:0/784097491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 msgr2=0x5601881e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 -- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 msgr2=0x56018823e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x56018823e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.818+0000 7f815579b700 1 -- 172.21.15.6:0/784097491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601880ed140 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f815579b700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d950 tx=0x7f814c0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b4d7d332b919e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f8138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0b9530 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f814c098ef0 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f814c0a1e50 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f814c09da30 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f816b422b80 10 client.15231 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560187f3dc10 con 0x5601881d5bb0 2023-11-17T00:08:19.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.819+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c0a1e50 con 0x5601881d5bb0 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f814c098960 con 0x5601881d5bb0 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 20 client.15231 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560188242d60 0x56018823f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 4 client.15231 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601880f5610 con 0x560188242d60 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.820+0000 7f8154f9a700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560188242d60 0x56018823f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f8154f9a700 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560188242d60 0x56018823f760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8cb2497dea8b21 server_cookie=339fafbe2317bf2c in_seq=0 out_seq=0 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f8138ff9700 10 client.15231 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f8138ff9700 1 -- 172.21.15.6:0/784097491 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5601880f5610 con 0x560188242d60 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f8138ff9700 10 client.15231 handle_command_reply: tid=0 2023-11-17T00:08:19.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 client.15231 shutdown 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 msgr2=0x5601881e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d950 tx=0x7f814c0076d0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 shutdown_connections 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560188242d60 0x56018823f760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5601881d5bb0 0x5601881e3600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5601881d70c0 0x56018823e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 --2- 172.21.15.6:0/784097491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5601881d6500 0x5601881e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 >> 172.21.15.6:0/784097491 conn(0x5601881cb310 msgr2=0x5601881e9870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f8138ff9700 0 client.15231 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 shutdown_connections 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 1 -- 172.21.15.6:0/784097491 wait complete. 2023-11-17T00:08:19.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.821+0000 7f816b422b80 20 client.15231 trim_cache size 0 max 16384 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dbc2f8cc-84db-11ee-8dbe-212e2dc638e7", 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 24578, 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 127.219598502, 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 20, 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 92, 2023-11-17T00:08:19.832 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 88, 2023-11-17T00:08:19.833 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 139.38005019299999 2023-11-17T00:08:19.833 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:08:19.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 msgr2=0x7f813c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 0x7f813c0a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f814800b830 tx=0x7f8148007740 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 msgr2=0x7f81501982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d980 tx=0x7f814c00dc90 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 shutdown_connections 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f813c0a11d0 0x7f813c0a3680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8150100240 0x7f81501982f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8150101dc0 0x7f815019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 --2- 172.21.15.6:0/1360620117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8150101430 0x7f8150198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 >> 172.21.15.6:0/1360620117 conn(0x7f81500fb820 msgr2=0x7f8150106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:19.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 shutdown_connections 2023-11-17T00:08:19.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:19.822+0000 7f81579ff700 1 -- 172.21.15.6:0/1360620117 wait complete. 2023-11-17T00:08:19.842 INFO:teuthology.run_tasks:Running task kclient... 2023-11-17T00:08:19.859 INFO:tasks.kclient:Mounting kernel clients... 2023-11-17T00:08:19.860 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-17T00:08:19.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:19.860 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-17T00:08:19.861 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3809011859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:19.861 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2564616127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:19.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3809011859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:19.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2564616127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:19.907 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-17T00:08:19.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:19.908 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-17T00:08:20.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:20.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-17T00:08:20.138 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-11-17T00:08:20.138 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-17T00:08:20.138 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:20.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph fs ls 2023-11-17T00:08:20.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:20 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3858224292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:20.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:20 smithi006 ceph-mon[97840]: pgmap v361: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:20 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3858224292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:20 smithi087 ceph-mon[131182]: pgmap v361: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:20.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:20 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3858224292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:20.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:20 smithi186 ceph-mon[131487]: pgmap v361: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:21.013 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:08:22.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1220496947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc1055e0 msgr2=0x7fdbbc10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:22.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1220496947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc10d160 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400b0d0 tx=0x7fdbb400b3e0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1220496947 shutdown_connections 2023-11-17T00:08:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1220496947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc10d160 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1220496947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1220496947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc100ee0 0x7fdbbc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1220496947 >> 172.21.15.6:0/1220496947 conn(0x7fdbbc0fbdc0 msgr2=0x7fdbbc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.907+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1220496947 shutdown_connections 2023-11-17T00:08:22.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.908+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1220496947 wait complete. 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.909+0000 7fdbc30d9700 1 Processor -- start 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.909+0000 7fdbc30d9700 1 -- start start 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc100ee0 0x7fdbbc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc198b50 con 0x7fdbbc100ee0 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc198c90 con 0x7fdbbc1055e0 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc30d9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc198e50 con 0x7fdbbc100540 2023-11-17T00:08:22.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59888/0 (socket says 172.21.15.6:59888) 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 -- 172.21.15.6:0/1260040621 learned_addr learned my addr 172.21.15.6:0/1260040621 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbbbfff700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc100ee0 0x7fdbbc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc1676700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 -- 172.21.15.6:0/1260040621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc1055e0 msgr2=0x7fdbbc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 -- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc100ee0 msgr2=0x7fdbbc19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc100ee0 0x7fdbbc19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.910+0000 7fdbc0e75700 1 -- 172.21.15.6:0/1260040621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4009d00 con 0x7fdbbc100540 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbc0e75700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d950 tx=0x7fdbb000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:22.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0009940 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb0010460 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000f620 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc110370 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.911+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc1101e0 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.912+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc110500 con 0x7fdbbc100540 2023-11-17T00:08:22.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.914+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdbb00108d0 con 0x7fdbbc100540 2023-11-17T00:08:22.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.916+0000 7fdbb9ffb700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 0x7fdbac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:22.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.916+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdbb00ccb50 con 0x7fdbbc100540 2023-11-17T00:08:22.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.916+0000 7fdbbbfff700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 0x7fdbac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:22.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.917+0000 7fdbbbfff700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 0x7fdbac0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc199ce0 tx=0x7fdba8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:22.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:22.920+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbb0095ed0 con 0x7fdbbc100540 2023-11-17T00:08:22.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:22 smithi006 ceph-mon[97840]: pgmap v362: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:22 smithi087 ceph-mon[131182]: pgmap v362: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:22 smithi186 ceph-mon[131487]: pgmap v362: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:23.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.171+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdbbc066ea0 con 0x7fdbbc100540 2023-11-17T00:08:23.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.172+0000 7fdbb9ffb700 1 -- 172.21.15.6:0/1260040621 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdbb0016070 con 0x7fdbbc100540 2023-11-17T00:08:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 msgr2=0x7fdbac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 0x7fdbac0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc199ce0 tx=0x7fdba8009260 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 msgr2=0x7fdbbc19e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d950 tx=0x7fdbb000dc60 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 shutdown_connections 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbac0a1150 0x7fdbac0a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc1055e0 0x7fdbbc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbbc100540 0x7fdbbc19e410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 --2- 172.21.15.6:0/1260040621 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbbc100ee0 0x7fdbbc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:23.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 >> 172.21.15.6:0/1260040621 conn(0x7fdbbc0fbdc0 msgr2=0x7fdbbc10b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:23.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 shutdown_connections 2023-11-17T00:08:23.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:23.174+0000 7fdbc30d9700 1 -- 172.21.15.6:0/1260040621 wait complete. 2023-11-17T00:08:23.898 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-17T00:08:23.899 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-17T00:08:23.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:23.899 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-17T00:08:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-17T00:08:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:08:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 41522sec preferred_lft 41522sec 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-17T00:08:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2023-11-17T00:08:23.949 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-17T00:08:23.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-17T00:08:23.949 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:08:24.092 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:23 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1260040621' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:08:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:23 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1260040621' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:08:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:23 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1260040621' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-17T00:08:24.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:24.196 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-17T00:08:24.266 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-17T00:08:24.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:24.270 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-17T00:08:24.317 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-17T00:08:24.317 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-17T00:08:24.318 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-17T00:08:24.318 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-17T00:08:24.318 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-17T00:08:24.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-17T00:08:24.319 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:08:24.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:24.499 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-17T00:08:24.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:24.544 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-17T00:08:24.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:31.633 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:08:31.633 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:08:31.633 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-17T00:08:31.633 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-17T00:08:31.633 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:08:31.635 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:24 smithi186 ceph-mon[131487]: pgmap v363: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.635 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:26 smithi186 ceph-mon[131487]: pgmap v364: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:28 smithi186 ceph-mon[131487]: pgmap v365: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:28 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:30 smithi186 ceph-mon[131487]: pgmap v366: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:24 smithi087 ceph-mon[131182]: pgmap v363: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:26 smithi087 ceph-mon[131182]: pgmap v364: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:28 smithi087 ceph-mon[131182]: pgmap v365: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:28 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:30 smithi087 ceph-mon[131182]: pgmap v366: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:24 smithi006 ceph-mon[97840]: pgmap v363: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.636 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:26 smithi006 ceph-mon[97840]: pgmap v364: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:28 smithi006 ceph-mon[97840]: pgmap v365: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:31.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:28 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:31.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:30 smithi006 ceph-mon[97840]: pgmap v366: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:08:31.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-17T00:08:31.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-17T00:08:31.792 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:08:32.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:32.066 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-17T00:08:32.067 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-17T00:08:32.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-17T00:08:32.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2023-11-17T00:08:32.067 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-17T00:08:32.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-17T00:08:32.254 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:32.302 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-17T00:08:32.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-17T00:08:32.302 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:32.346 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-17T00:08:32.346 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-17T00:08:32.346 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/ad45d892-14a7-476f-b7ff-60e41d566f52 2023-11-17T00:08:32.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:32.347 DEBUG:teuthology.orchestra.run.smithi006:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/ad45d892-14a7-476f-b7ff-60e41d566f52 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-11-17T00:08:32.502 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-11-17T00:08:32.503 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-11-17T00:08:32.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:32 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:32 smithi087 ceph-mon[131182]: pgmap v367: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:32.605 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.605 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.605 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.606 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:32 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.606 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:32 smithi006 ceph-mon[97840]: pgmap v367: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:32.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:32 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:32.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:32 smithi186 ceph-mon[131487]: pgmap v367: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:08:32.966 INFO:teuthology.orchestra.run.smithi006.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-11-17T00:08:32.966 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-11-17T00:08:32.966 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: resolved to: "172.21.15.6:3300,172.21.15.87:3300,172.21.15.186:3300" 2023-11-17T00:08:32.966 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: trying mount with new device syntax: 0@dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/ad45d892-14a7-476f-b7ff-60e41d566f52 2023-11-17T00:08:32.967 INFO:teuthology.orchestra.run.smithi006.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.6:3300/172.21.15.87:3300/172.21.15.186:3300" will pass to kernel 2023-11-17T00:08:32.980 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-17T00:08:32.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:32.981 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:33.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:33.055 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-17T00:08:33.148 INFO:teuthology.orchestra.run.smithi006.stdout:client25267 2023-11-17T00:08:33.149 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:08:33.149 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-17T00:08:33.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:33.194 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-17T00:08:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:client25267 2023-11-17T00:08:33.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:33.327 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-17T00:08:33.423 INFO:teuthology.orchestra.run.smithi006.stdout:client25267 2023-11-17T00:08:33.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:08:33.423 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7.client25267/status') 2023-11-17T00:08:33.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:33.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:33 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:33.536 INFO:teuthology.orchestra.run.smithi006.stdout:instance: client.25267 (3)172.21.15.6:0/463581782 2023-11-17T00:08:33.536 INFO:teuthology.orchestra.run.smithi006.stdout:blocklisted: false 2023-11-17T00:08:33.537 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records in 2023-11-17T00:08:33.537 INFO:teuthology.orchestra.run.smithi006.stderr:0+1 records out 2023-11-17T00:08:33.537 INFO:teuthology.orchestra.run.smithi006.stderr:69 bytes copied, 5.6833e-05 s, 1.2 MB/s 2023-11-17T00:08:33.540 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-17T00:08:33.561 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-11-17T00:08:33.577 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 -- 172.21.15.6:0/2998097032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa81002d0 msgr2=0x7ffaa81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 --2- 172.21.15.6:0/2998097032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa81002d0 0x7ffaa81006e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009990 tx=0x7ffaa4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 -- 172.21.15.6:0/2998097032 shutdown_connections 2023-11-17T00:08:33.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 --2- 172.21.15.6:0/2998097032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa81002d0 0x7ffaa81006e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 --2- 172.21.15.6:0/2998097032 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa8101e50 0x7ffaa8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 --2- 172.21.15.6:0/2998097032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81014c0 0x7ffaa8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 -- 172.21.15.6:0/2998097032 >> 172.21.15.6:0/2998097032 conn(0x7ffaa80fb8b0 msgr2=0x7ffaa80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 -- 172.21.15.6:0/2998097032 shutdown_connections 2023-11-17T00:08:33.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.786+0000 7ffab05ff700 1 -- 172.21.15.6:0/2998097032 wait complete. 2023-11-17T00:08:33.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 Processor -- start 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 -- start start 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 0x7ffaa81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa8101e50 0x7ffaa819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819fe00 con 0x7ffaa8101e50 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa819ff40 con 0x7ffaa81014c0 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffab05ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81a00d0 con 0x7ffaa81002d0 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffaaeb9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa8101e50 0x7ffaa819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.788+0000 7ffaae39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47360/0 (socket says 172.21.15.6:47360) 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 -- 172.21.15.6:0/3750897363 learned_addr learned my addr 172.21.15.6:0/3750897363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:33.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaadb9a700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 0x7ffaa81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 -- 172.21.15.6:0/3750897363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 msgr2=0x7ffaa81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 0x7ffaa81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 -- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa8101e50 msgr2=0x7ffaa819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa8101e50 0x7ffaa819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 -- 172.21.15.6:0/3750897363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40095f0 con 0x7ffaa81002d0 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaadb9a700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 0x7ffaa81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:33.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.789+0000 7ffaae39b700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004730 tx=0x7ffaa4004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.790+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa401d070 con 0x7ffaa81002d0 2023-11-17T00:08:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.790+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa810bca0 con 0x7ffaa81002d0 2023-11-17T00:08:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.790+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa810bb10 con 0x7ffaa81002d0 2023-11-17T00:08:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.790+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaa4021410 con 0x7ffaa81002d0 2023-11-17T00:08:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.790+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4017650 con 0x7ffaa81002d0 2023-11-17T00:08:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.791+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8192a00 con 0x7ffaa81002d0 2023-11-17T00:08:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.793+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffaa4021580 con 0x7ffaa81002d0 2023-11-17T00:08:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.794+0000 7ffa9f7fe700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 0x7ffa940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:33.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.794+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffaa40ce1c0 con 0x7ffaa81002d0 2023-11-17T00:08:33.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.794+0000 7ffaadb9a700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 0x7ffa940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:33.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.795+0000 7ffaadb9a700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 0x7ffa940a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011490 tx=0x7ffa98009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:33.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:33.796+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffaa40975d0 con 0x7ffaa81002d0 2023-11-17T00:08:33.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:33.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:33 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:33.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:33.873 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:33 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:34.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.011+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa8061700 con 0x7ffaa81002d0 2023-11-17T00:08:34.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.013+0000 7ffa9f7fe700 1 -- 172.21.15.6:0/3750897363 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7ffaa409abf0 con 0x7ffaa81002d0 2023-11-17T00:08:34.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:34.015 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 msgr2=0x7ffa940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 0x7ffa940a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011490 tx=0x7ffa98009260 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 msgr2=0x7ffaa8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004730 tx=0x7ffaa4004760 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 shutdown_connections 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7ffa940a1150 0x7ffa940a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa8101e50 0x7ffaa819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffaa81002d0 0x7ffaa8198280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 --2- 172.21.15.6:0/3750897363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffaa81014c0 0x7ffaa81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.015+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 >> 172.21.15.6:0/3750897363 conn(0x7ffaa80fb8b0 msgr2=0x7ffaa8106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.016+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 shutdown_connections 2023-11-17T00:08:34.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.016+0000 7ffab05ff700 1 -- 172.21.15.6:0/3750897363 wait complete. 2023-11-17T00:08:34.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:08:34.038 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:08:34.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 -- 172.21.15.6:0/1119523954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94100290 msgr2=0x7fbd941006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 --2- 172.21.15.6:0/1119523954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94100290 0x7fbd941006a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009990 tx=0x7fbd84009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 -- 172.21.15.6:0/1119523954 shutdown_connections 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 --2- 172.21.15.6:0/1119523954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101e10 0x7fbd941086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 --2- 172.21.15.6:0/1119523954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94100290 0x7fbd941006a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 --2- 172.21.15.6:0/1119523954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94101480 0x7fbd941018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.247+0000 7fbd999ff700 1 -- 172.21.15.6:0/1119523954 >> 172.21.15.6:0/1119523954 conn(0x7fbd940fb870 msgr2=0x7fbd940fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.248+0000 7fbd999ff700 1 -- 172.21.15.6:0/1119523954 shutdown_connections 2023-11-17T00:08:34.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.248+0000 7fbd999ff700 1 -- 172.21.15.6:0/1119523954 wait complete. 2023-11-17T00:08:34.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.249+0000 7fbd999ff700 1 Processor -- start 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.249+0000 7fbd999ff700 1 -- start start 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 0x7fbd94196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94101e10 0x7fbd9419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9419dbb0 con 0x7fbd94101480 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9419dcf0 con 0x7fbd94101e10 2023-11-17T00:08:34.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd999ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9419de80 con 0x7fbd94100290 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd92ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd927fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 0x7fbd94196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd92ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47376/0 (socket says 172.21.15.6:47376) 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd92ffd700 1 -- 172.21.15.6:0/931139519 learned_addr learned my addr 172.21.15.6:0/931139519 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd937fe700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94101e10 0x7fbd9419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.250+0000 7fbd92ffd700 1 -- 172.21.15.6:0/931139519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94101e10 msgr2=0x7fbd9419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd92ffd700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94101e10 0x7fbd9419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd92ffd700 1 -- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 msgr2=0x7fbd94196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd92ffd700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 0x7fbd94196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd92ffd700 1 -- 172.21.15.6:0/931139519 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009c40 con 0x7fbd94100290 2023-11-17T00:08:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd927fc700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 0x7fbd94196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:34.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd92ffd700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005740 tx=0x7fbd84005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8401d070 con 0x7fbd94100290 2023-11-17T00:08:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd840095f0 con 0x7fbd94100290 2023-11-17T00:08:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd84021410 con 0x7fbd94100290 2023-11-17T00:08:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84017650 con 0x7fbd94100290 2023-11-17T00:08:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.251+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9410bc40 con 0x7fbd94100290 2023-11-17T00:08:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.252+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9410bdd0 con 0x7fbd94100290 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.255+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbd84021580 con 0x7fbd94100290 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.256+0000 7fbd989fd700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.256+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbd840ce730 con 0x7fbd94100290 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.256+0000 7fbd927fc700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.257+0000 7fbd927fc700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011490 tx=0x7fbd88009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.258+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd84097b40 con 0x7fbd94100290 2023-11-17T00:08:34.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.467+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd94107280 con 0x7fbd94100290 2023-11-17T00:08:34.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.468+0000 7fbd989fd700 1 -- 172.21.15.6:0/931139519 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd8409b160 con 0x7fbd94100290 2023-11-17T00:08:34.469 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:08:34.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 msgr2=0x7fbd7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011490 tx=0x7fbd88009260 comp rx=0 tx=0).stop 2023-11-17T00:08:34.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 msgr2=0x7fbd94196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005740 tx=0x7fbd84005770 comp rx=0 tx=0).stop 2023-11-17T00:08:34.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 shutdown_connections 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd94101e10 0x7fbd9419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd94100290 0x7fbd94196030 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 --2- 172.21.15.6:0/931139519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd94101480 0x7fbd94196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 >> 172.21.15.6:0/931139519 conn(0x7fbd940fb870 msgr2=0x7fbd940fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 shutdown_connections 2023-11-17T00:08:34.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.471+0000 7fbd999ff700 1 -- 172.21.15.6:0/931139519 wait complete. 2023-11-17T00:08:34.491 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 -- 172.21.15.6:0/1019098772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548101e10 msgr2=0x7f15481086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 --2- 172.21.15.6:0/1019098772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548101e10 0x7f15481086d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f154000b0a0 tx=0x7f154000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 -- 172.21.15.6:0/1019098772 shutdown_connections 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 --2- 172.21.15.6:0/1019098772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548101e10 0x7f15481086d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 --2- 172.21.15.6:0/1019098772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548100290 0x7f15481006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 --2- 172.21.15.6:0/1019098772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 0x7f15481018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 -- 172.21.15.6:0/1019098772 >> 172.21.15.6:0/1019098772 conn(0x7f15480fb870 msgr2=0x7f15480fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:34.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 -- 172.21.15.6:0/1019098772 shutdown_connections 2023-11-17T00:08:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.700+0000 7f15503ff700 1 -- 172.21.15.6:0/1019098772 wait complete. 2023-11-17T00:08:34.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 Processor -- start 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 -- start start 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 0x7f1548072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548101e10 0x7f1548073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548073bc0 con 0x7f1548100290 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548073d30 con 0x7f1548101480 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f15503ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481a0160 con 0x7f1548101e10 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.701+0000 7f154e19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35828/0 (socket says 172.21.15.6:35828) 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 -- 172.21.15.6:0/1356228003 learned_addr learned my addr 172.21.15.6:0/1356228003 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154d99a700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 0x7f1548072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e99c700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548101e10 0x7f1548073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 -- 172.21.15.6:0/1356228003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548101e10 msgr2=0x7f1548073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548101e10 0x7f1548073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 -- 172.21.15.6:0/1356228003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 msgr2=0x7f1548072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 0x7f1548072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 -- 172.21.15.6:0/1356228003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009520 con 0x7f1548100290 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f154e19b700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f154400ea50 tx=0x7f154400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400cbd0 con 0x7f1548100290 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154400cd30 con 0x7f1548100290 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.702+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544018680 con 0x7f1548100290 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.703+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1538009520 con 0x7f1548100290 2023-11-17T00:08:34.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.703+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1540009d00 con 0x7f1548100290 2023-11-17T00:08:34.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.704+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1548107840 con 0x7f1548100290 2023-11-17T00:08:34.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.706+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1544010430 con 0x7f1548100290 2023-11-17T00:08:34.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.707+0000 7f153f7fe700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 0x7f15340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:34.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.707+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1544014070 con 0x7f1548100290 2023-11-17T00:08:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.707+0000 7f154d99a700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 0x7f15340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.708+0000 7f154d99a700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 0x7f15340a3680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1538009df0 tx=0x7f1538009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:34.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.711+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1544096470 con 0x7f1548100290 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: pgmap v368: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:08:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:34 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3750897363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:08:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:08:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: pgmap v368: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:08:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:34 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3750897363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: from='mgr.14148 172.21.15.6:0/2131201522' entity='mgr.x' 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: pgmap v368: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:08:34.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:34 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3750897363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:34.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.921+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1548061700 con 0x7f1548100290 2023-11-17T00:08:34.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.922+0000 7f153f7fe700 1 -- 172.21.15.6:0/1356228003 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1544099a90 con 0x7f1548100290 2023-11-17T00:08:34.923 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:34.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 msgr2=0x7f15340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 0x7f15340a3680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1538009df0 tx=0x7f1538009260 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 msgr2=0x7f15480748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f154400ea50 tx=0x7f154400ed60 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 shutdown_connections 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f15340a11d0 0x7f15340a3680 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1548100290 0x7f15480748a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1548101e10 0x7f1548073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.924+0000 7f15503ff700 1 --2- 172.21.15.6:0/1356228003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1548101480 0x7f1548072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.925+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 >> 172.21.15.6:0/1356228003 conn(0x7f15480fb870 msgr2=0x7f1548104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.925+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 shutdown_connections 2023-11-17T00:08:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:34.925+0000 7f15503ff700 1 -- 172.21.15.6:0/1356228003 wait complete. 2023-11-17T00:08:34.944 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:08:35.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 -- 172.21.15.6:0/1188956491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac101e10 msgr2=0x7f72ac1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1188956491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac101e10 0x7f72ac1086d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b0a0 tx=0x7f72a400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 -- 172.21.15.6:0/1188956491 shutdown_connections 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1188956491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac101e10 0x7f72ac1086d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1188956491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac100290 0x7f72ac1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1188956491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 0x7f72ac1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 -- 172.21.15.6:0/1188956491 >> 172.21.15.6:0/1188956491 conn(0x7f72ac0fb870 msgr2=0x7f72ac0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 -- 172.21.15.6:0/1188956491 shutdown_connections 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.154+0000 7f72b1fff700 1 -- 172.21.15.6:0/1188956491 wait complete. 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 Processor -- start 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 -- start start 2023-11-17T00:08:35.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 0x7f72ac198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac101e10 0x7f72ac19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac19fe70 con 0x7f72ac100290 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac19ffb0 con 0x7f72ac101e10 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.155+0000 7f72b1fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac1a0140 con 0x7f72ac101480 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35842/0 (socket says 172.21.15.6:35842) 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 -- 172.21.15.6:0/1508233184 learned_addr learned my addr 172.21.15.6:0/1508233184 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72a3fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 0x7f72ac198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72abfff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac101e10 0x7f72ac19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 -- 172.21.15.6:0/1508233184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 msgr2=0x7f72ac198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 0x7f72ac198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 -- 172.21.15.6:0/1508233184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac101e10 msgr2=0x7f72ac19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac101e10 0x7f72ac19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 -- 172.21.15.6:0/1508233184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009d00 con 0x7f72ac100290 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72ab7fe700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f729c00e970 tx=0x7f729c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.156+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c009790 con 0x7f72ac100290 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.157+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729c00c7e0 con 0x7f72ac100290 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.157+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac10bc70 con 0x7f72ac100290 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.157+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c003c70 con 0x7f72ac100290 2023-11-17T00:08:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.157+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac10bae0 con 0x7f72ac100290 2023-11-17T00:08:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.158+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac1928e0 con 0x7f72ac100290 2023-11-17T00:08:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.160+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f729c00cc50 con 0x7f72ac100290 2023-11-17T00:08:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.162+0000 7f72a97fa700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 0x7f72980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.162+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f729c014070 con 0x7f72ac100290 2023-11-17T00:08:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.162+0000 7f72a3fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 0x7f72980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.163+0000 7f72a3fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 0x7f72980a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7294009c60 tx=0x7f7294009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:35.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.166+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f729c096210 con 0x7f72ac100290 2023-11-17T00:08:35.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.378+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72ac061700 con 0x7f72ac100290 2023-11-17T00:08:35.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.379+0000 7f72a97fa700 1 -- 172.21.15.6:0/1508233184 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f729c003dd0 con 0x7f72ac100290 2023-11-17T00:08:35.379 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 msgr2=0x7f72980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 0x7f72980a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7294009c60 tx=0x7f7294009260 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 msgr2=0x7f72ac1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f729c00e970 tx=0x7f729c00ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 shutdown_connections 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f72980a1150 0x7f72980a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72ac100290 0x7f72ac1982f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72ac101e10 0x7f72ac19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 --2- 172.21.15.6:0/1508233184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72ac101480 0x7f72ac198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 >> 172.21.15.6:0/1508233184 conn(0x7f72ac0fb870 msgr2=0x7f72ac106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 shutdown_connections 2023-11-17T00:08:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.381+0000 7f72b1fff700 1 -- 172.21.15.6:0/1508233184 wait complete. 2023-11-17T00:08:35.401 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:08:35.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 -- 172.21.15.6:0/950698086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101490 msgr2=0x7f85641018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 --2- 172.21.15.6:0/950698086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101490 0x7f85641018e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8558009990 tx=0x7f8558009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 -- 172.21.15.6:0/950698086 shutdown_connections 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 --2- 172.21.15.6:0/950698086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101e20 0x7f85641086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 --2- 172.21.15.6:0/950698086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 --2- 172.21.15.6:0/950698086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101490 0x7f85641018e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 -- 172.21.15.6:0/950698086 >> 172.21.15.6:0/950698086 conn(0x7f85640fb880 msgr2=0x7f85640fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.609+0000 7f8569dff700 1 -- 172.21.15.6:0/950698086 shutdown_connections 2023-11-17T00:08:35.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.610+0000 7f8569dff700 1 -- 172.21.15.6:0/950698086 wait complete. 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 Processor -- start 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 -- start start 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101490 0x7f8564198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 0x7f856419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856419fd60 con 0x7f8564101e20 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856419fea0 con 0x7f85641002a0 2023-11-17T00:08:35.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8569dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641a0030 con 0x7f8564101490 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8563fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 0x7f856419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f85637fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f8562ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101490 0x7f8564198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f85637fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57870/0 (socket says 172.21.15.6:57870) 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.611+0000 7f85637fe700 1 -- 172.21.15.6:0/1727016653 learned_addr learned my addr 172.21.15.6:0/1727016653 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 -- 172.21.15.6:0/1727016653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101490 msgr2=0x7f8564198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101490 0x7f8564198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 -- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 msgr2=0x7f856419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 0x7f856419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 -- 172.21.15.6:0/1727016653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580095f0 con 0x7f85641002a0 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f8563fff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 0x7f856419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:35.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f85637fe700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f855400e970 tx=0x7f855400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554009790 con 0x7f85641002a0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855400c7e0 con 0x7f85641002a0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554003c70 con 0x7f85641002a0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.612+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856410bc50 con 0x7f85641002a0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.613+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856410bac0 con 0x7f85641002a0 2023-11-17T00:08:35.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.613+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85641a0730 con 0x7f85641002a0 2023-11-17T00:08:35.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.615+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f855400cc50 con 0x7f85641002a0 2023-11-17T00:08:35.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.617+0000 7f8560ff9700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:35.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.617+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8554014070 con 0x7f85641002a0 2023-11-17T00:08:35.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.617+0000 7f8562ffd700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:35.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.618+0000 7f8562ffd700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 0x7f854c0a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f855800b2e0 tx=0x7f8558005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:35.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.621+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8554096170 con 0x7f85641002a0 2023-11-17T00:08:35.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:35 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/931139519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:35 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1356228003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:35 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1508233184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.832+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f85641a0ce0 con 0x7f85641002a0 2023-11-17T00:08:35.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.833+0000 7f8560ff9700 1 -- 172.21.15.6:0/1727016653 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8554003dd0 con 0x7f85641002a0 2023-11-17T00:08:35.834 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 msgr2=0x7f854c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 0x7f854c0a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f855800b2e0 tx=0x7f8558005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 msgr2=0x7f85641981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f855400e970 tx=0x7f855400ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 shutdown_connections 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8564101e20 0x7f856419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85641002a0 0x7f85641981e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 --2- 172.21.15.6:0/1727016653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8564101490 0x7f8564198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 >> 172.21.15.6:0/1727016653 conn(0x7f85640fb880 msgr2=0x7f85640fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 shutdown_connections 2023-11-17T00:08:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:35.836+0000 7f8569dff700 1 -- 172.21.15.6:0/1727016653 wait complete. 2023-11-17T00:08:35.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:35 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/931139519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:35 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1356228003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:35 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1508233184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.857 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:35.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:35 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/931139519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:35 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1356228003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:35.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:35 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1508233184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 -- 172.21.15.6:0/3223381236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01014c0 msgr2=0x7f57b0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 --2- 172.21.15.6:0/3223381236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01014c0 0x7f57b0101910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f579c009990 tx=0x7f579c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 -- 172.21.15.6:0/3223381236 shutdown_connections 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 --2- 172.21.15.6:0/3223381236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b0101e50 0x7f57b0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 --2- 172.21.15.6:0/3223381236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b01002d0 0x7f57b01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 --2- 172.21.15.6:0/3223381236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01014c0 0x7f57b0101910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 -- 172.21.15.6:0/3223381236 >> 172.21.15.6:0/3223381236 conn(0x7f57b00fb8b0 msgr2=0x7f57b00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 -- 172.21.15.6:0/3223381236 shutdown_connections 2023-11-17T00:08:36.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.061+0000 7f57b6fff700 1 -- 172.21.15.6:0/3223381236 wait complete. 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.062+0000 7f57b6fff700 1 Processor -- start 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.062+0000 7f57b6fff700 1 -- start start 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.062+0000 7f57b6fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 0x7f57b0196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.062+0000 7f57b6fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b6fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b0101e50 0x7f57b019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b6fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019dc00 con 0x7f57b01014c0 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b6fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019dd40 con 0x7f57b01002d0 2023-11-17T00:08:36.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b6fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019ded0 con 0x7f57b0101e50 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35866/0 (socket says 172.21.15.6:35866) 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 -- 172.21.15.6:0/2305422252 learned_addr learned my addr 172.21.15.6:0/2305422252 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b4d9b700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 0x7f57b0196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b559c700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b0101e50 0x7f57b019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 -- 172.21.15.6:0/2305422252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b0101e50 msgr2=0x7f57b019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b0101e50 0x7f57b019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 -- 172.21.15.6:0/2305422252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 msgr2=0x7f57b0196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 0x7f57b0196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57affff700 1 -- 172.21.15.6:0/2305422252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0095f0 con 0x7f57b01014c0 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.063+0000 7f57b4d9b700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 0x7f57b0196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57affff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f579c004760 tx=0x7f579c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c01d070 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579c004830 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b010bc80 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c005350 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.064+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b010baf0 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.065+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b0107860 con 0x7f57b01014c0 2023-11-17T00:08:36.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.067+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f579c0054f0 con 0x7f57b01014c0 2023-11-17T00:08:36.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.068+0000 7f57adffb700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 0x7f57a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.069+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f579c0ce330 con 0x7f57b01014c0 2023-11-17T00:08:36.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.069+0000 7f57b4d9b700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 0x7f57a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.069+0000 7f57b4d9b700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 0x7f57a00a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57b0101320 tx=0x7f57a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.070+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f579c097740 con 0x7f57b01014c0 2023-11-17T00:08:36.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.284+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57b0067af0 con 0x7f57b01014c0 2023-11-17T00:08:36.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.285+0000 7f57adffb700 1 -- 172.21.15.6:0/2305422252 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7f579c09ad60 con 0x7f57b01014c0 2023-11-17T00:08:36.287 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:36.287 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:36.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.287+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 msgr2=0x7f57a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 0x7f57a00a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f57b0101320 tx=0x7f57a4009260 comp rx=0 tx=0).stop 2023-11-17T00:08:36.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 msgr2=0x7f57b01965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f579c004760 tx=0x7f579c004790 comp rx=0 tx=0).stop 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 shutdown_connections 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f57a00a1150 0x7f57a00a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57b0101e50 0x7f57b019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b01002d0 0x7f57b0196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 --2- 172.21.15.6:0/2305422252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57b01014c0 0x7f57b01965c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 >> 172.21.15.6:0/2305422252 conn(0x7f57b00fb8b0 msgr2=0x7f57b01049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:36.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 shutdown_connections 2023-11-17T00:08:36.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.288+0000 7f57b6fff700 1 -- 172.21.15.6:0/2305422252 wait complete. 2023-11-17T00:08:36.308 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:08:36.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 -- 172.21.15.6:0/3940182400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101480 msgr2=0x7fc7ac1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/3940182400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101480 0x7fc7ac1018d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098f0 tx=0x7fc7a0009c00 comp rx=0 tx=0).stop 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 -- 172.21.15.6:0/3940182400 shutdown_connections 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/3940182400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac101e10 0x7fc7ac1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/3940182400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac100290 0x7fc7ac1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/3940182400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101480 0x7fc7ac1018d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 -- 172.21.15.6:0/3940182400 >> 172.21.15.6:0/3940182400 conn(0x7fc7ac0fb870 msgr2=0x7fc7ac0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 -- 172.21.15.6:0/3940182400 shutdown_connections 2023-11-17T00:08:36.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.518+0000 7fc7b29ff700 1 -- 172.21.15.6:0/3940182400 wait complete. 2023-11-17T00:08:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 Processor -- start 2023-11-17T00:08:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 -- start start 2023-11-17T00:08:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac101480 0x7fc7ac072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 0x7fc7ac073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac073bc0 con 0x7fc7ac100290 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac073d30 con 0x7fc7ac101e10 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.519+0000 7fc7b29ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1a00d0 con 0x7fc7ac101480 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35894/0 (socket says 172.21.15.6:35894) 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 -- 172.21.15.6:0/2894783574 learned_addr learned my addr 172.21.15.6:0/2894783574 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7ab7fe700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac101480 0x7fc7ac072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7b0f9c700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 0x7fc7ac073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 -- 172.21.15.6:0/2894783574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac101480 msgr2=0x7fc7ac072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac101480 0x7fc7ac072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 -- 172.21.15.6:0/2894783574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 msgr2=0x7fc7ac073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 0x7fc7ac073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 -- 172.21.15.6:0/2894783574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009520 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7b0f9c700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 0x7fc7ac073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7abfff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ea50 tx=0x7fc79c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.520+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00cbd0 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.521+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4009d00 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.521+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a0009520 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.521+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79c00cd30 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.521+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c018680 con 0x7fc7ac100290 2023-11-17T00:08:36.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.522+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc798005310 con 0x7fc7ac100290 2023-11-17T00:08:36.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.524+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc79c0187e0 con 0x7fc7ac100290 2023-11-17T00:08:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.525+0000 7fc7a97fa700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 0x7fc7940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.525+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc79c014070 con 0x7fc7ac100290 2023-11-17T00:08:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.525+0000 7fc7ab7fe700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 0x7fc7940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.526+0000 7fc7ab7fe700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 0x7fc7940a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005e90 tx=0x7fc7a0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.527+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc79c0968b0 con 0x7fc7ac100290 2023-11-17T00:08:36.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:36 smithi006 ceph-mon[97840]: pgmap v369: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:08:36.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:36 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1727016653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:36.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:36 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2305422252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:36.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.741+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc798005f60 con 0x7fc7ac100290 2023-11-17T00:08:36.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.741+0000 7fc7a97fa700 1 -- 172.21.15.6:0/2894783574 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc79c099ed0 con 0x7fc7ac100290 2023-11-17T00:08:36.742 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 msgr2=0x7fc7940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 0x7fc7940a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0005e90 tx=0x7fc7a0019040 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 msgr2=0x7fc7ac0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00ea50 tx=0x7fc79c00ed60 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 shutdown_connections 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fc7940a1150 0x7fc7940a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7ac100290 0x7fc7ac0748a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7ac101e10 0x7fc7ac073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 --2- 172.21.15.6:0/2894783574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7ac101480 0x7fc7ac072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 >> 172.21.15.6:0/2894783574 conn(0x7fc7ac0fb870 msgr2=0x7fc7ac0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:36.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 shutdown_connections 2023-11-17T00:08:36.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.744+0000 7fc7b29ff700 1 -- 172.21.15.6:0/2894783574 wait complete. 2023-11-17T00:08:36.764 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:08:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:36 smithi087 ceph-mon[131182]: pgmap v369: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:08:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:36 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1727016653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:36 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2305422252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:36 smithi186 ceph-mon[131487]: pgmap v369: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:08:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:36 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1727016653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:36 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2305422252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 -- 172.21.15.6:0/2743149637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b410c870 msgr2=0x7f31b410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/2743149637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b410c870 0x7f31b410ec50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b0a0 tx=0x7f31ac00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 -- 172.21.15.6:0/2743149637 shutdown_connections 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/2743149637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b410c870 0x7f31b410ec50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/2743149637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4074b30 0x7f31b4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/2743149637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b4073550 0x7f31b40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 -- 172.21.15.6:0/2743149637 >> 172.21.15.6:0/2743149637 conn(0x7f31b40fb880 msgr2=0x7f31b40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 -- 172.21.15.6:0/2743149637 shutdown_connections 2023-11-17T00:08:36.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.977+0000 7f31bb1ff700 1 -- 172.21.15.6:0/2743149637 wait complete. 2023-11-17T00:08:36.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 Processor -- start 2023-11-17T00:08:36.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 -- start start 2023-11-17T00:08:36.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b410c870 0x7f31b41a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a4260 con 0x7f31b4074b30 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a43a0 con 0x7f31b410c870 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.978+0000 7f31bb1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a4530 con 0x7f31b4073550 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47492/0 (socket says 172.21.15.6:47492) 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 -- 172.21.15.6:0/488143600 learned_addr learned my addr 172.21.15.6:0/488143600 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b3fff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35914/0 (socket says 172.21.15.6:35914) 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b979c700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b410c870 0x7f31b41a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 -- 172.21.15.6:0/488143600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b410c870 msgr2=0x7f31b41a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b410c870 0x7f31b41a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 -- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 msgr2=0x7f31b419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 -- 172.21.15.6:0/488143600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac009d00 con 0x7f31b4073550 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b3fff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.979+0000 7f31b8f9b700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e970 tx=0x7f31a800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.980+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8009790 con 0x7f31b4073550 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.980+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b4112280 con 0x7f31b4073550 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.980+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31a800c7e0 con 0x7f31b4073550 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.980+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a80054e0 con 0x7f31b4073550 2023-11-17T00:08:36.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.980+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b41120f0 con 0x7f31b4073550 2023-11-17T00:08:36.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.981+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b41023c0 con 0x7f31b4073550 2023-11-17T00:08:36.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.983+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f31a800c950 con 0x7f31b4073550 2023-11-17T00:08:36.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.984+0000 7f31b1ffb700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 0x7f31a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:36.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.984+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31a8014070 con 0x7f31b4073550 2023-11-17T00:08:36.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.985+0000 7f31b3fff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 0x7f31a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:36.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.985+0000 7f31b3fff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 0x7f31a40a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31b4074850 tx=0x7f31a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:36.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:36.986+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31a80964e0 con 0x7f31b4073550 2023-11-17T00:08:37.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.197+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31b4061700 con 0x7f31b4073550 2023-11-17T00:08:37.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.198+0000 7f31b1ffb700 1 -- 172.21.15.6:0/488143600 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31a8099b00 con 0x7f31b4073550 2023-11-17T00:08:37.233 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:37.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.200+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 msgr2=0x7f31a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.200+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 0x7f31a40a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31b4074850 tx=0x7f31a0009260 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.200+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 msgr2=0x7f31b419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.200+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e970 tx=0x7f31a800ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 shutdown_connections 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f31a40a1150 0x7f31a40a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31b410c870 0x7f31b41a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31b4073550 0x7f31b419c6e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 --2- 172.21.15.6:0/488143600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b4074b30 0x7f31b419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 >> 172.21.15.6:0/488143600 conn(0x7f31b40fb880 msgr2=0x7f31b40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:37.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 shutdown_connections 2023-11-17T00:08:37.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.201+0000 7f31bb1ff700 1 -- 172.21.15.6:0/488143600 wait complete. 2023-11-17T00:08:37.269 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 -- 172.21.15.6:0/901709106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101e20 msgr2=0x7f3de01086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 --2- 172.21.15.6:0/901709106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101e20 0x7f3de01086e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009f90 tx=0x7f3dd800b330 comp rx=0 tx=0).stop 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 -- 172.21.15.6:0/901709106 shutdown_connections 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 --2- 172.21.15.6:0/901709106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101e20 0x7f3de01086e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 --2- 172.21.15.6:0/901709106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de01002a0 0x7f3de01006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 --2- 172.21.15.6:0/901709106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de0101490 0x7f3de01018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 -- 172.21.15.6:0/901709106 >> 172.21.15.6:0/901709106 conn(0x7f3de00fb880 msgr2=0x7f3de00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 -- 172.21.15.6:0/901709106 shutdown_connections 2023-11-17T00:08:37.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.481+0000 7f3de87ff700 1 -- 172.21.15.6:0/901709106 wait complete. 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.482+0000 7f3de87ff700 1 Processor -- start 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.482+0000 7f3de87ff700 1 -- start start 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.482+0000 7f3de87ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de01002a0 0x7f3de01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de87ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de87ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de0101e20 0x7f3de019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de87ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019dc60 con 0x7f3de0101490 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de87ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019dda0 con 0x7f3de01002a0 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de87ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019df30 con 0x7f3de0101e20 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35930/0 (socket says 172.21.15.6:35930) 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 -- 172.21.15.6:0/3321319028 learned_addr learned my addr 172.21.15.6:0/3321319028 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de6d9c700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de0101e20 0x7f3de019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de659b700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de01002a0 0x7f3de01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 -- 172.21.15.6:0/3321319028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de0101e20 msgr2=0x7f3de019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de0101e20 0x7f3de019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 -- 172.21.15.6:0/3321319028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de01002a0 msgr2=0x7f3de01960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de01002a0 0x7f3de01960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 -- 172.21.15.6:0/3321319028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009c40 con 0x7f3de0101490 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.483+0000 7f3de5d9a700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9c0 tx=0x7f3dd000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.484+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0004a30 con 0x7f3de0101490 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.484+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dd000cc80 con 0x7f3de0101490 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.484+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de010bc70 con 0x7f3de0101490 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.484+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00054e0 con 0x7f3de0101490 2023-11-17T00:08:37.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.484+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de010bae0 con 0x7f3de0101490 2023-11-17T00:08:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.485+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de01077e0 con 0x7f3de0101490 2023-11-17T00:08:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.487+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3dd0004b90 con 0x7f3de0101490 2023-11-17T00:08:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.488+0000 7f3dd77fe700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 0x7f3dcc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.489+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3dd0014070 con 0x7f3de0101490 2023-11-17T00:08:37.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.489+0000 7f3de659b700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 0x7f3dcc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.489+0000 7f3de659b700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 0x7f3dcc0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3de01012f0 tx=0x7f3ddc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:37.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.492+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3dd0096130 con 0x7f3de0101490 2023-11-17T00:08:37.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.704+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3de0061700 con 0x7f3de0101490 2023-11-17T00:08:37.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.705+0000 7f3dd77fe700 1 -- 172.21.15.6:0/3321319028 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3dd0099750 con 0x7f3de0101490 2023-11-17T00:08:37.706 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:08:37.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 msgr2=0x7f3dcc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 0x7f3dcc0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3de01012f0 tx=0x7f3ddc009260 comp rx=0 tx=0).stop 2023-11-17T00:08:37.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 msgr2=0x7f3de0196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000e9c0 tx=0x7f3dd000ecd0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 shutdown_connections 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f3dcc0a1150 0x7f3dcc0a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3de0101e20 0x7f3de019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de01002a0 0x7f3de01960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 --2- 172.21.15.6:0/3321319028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3de0101490 0x7f3de0196620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.707+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 >> 172.21.15.6:0/3321319028 conn(0x7f3de00fb880 msgr2=0x7f3de0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.708+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 shutdown_connections 2023-11-17T00:08:37.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.708+0000 7f3de87ff700 1 -- 172.21.15.6:0/3321319028 wait complete. 2023-11-17T00:08:37.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2894783574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:37 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/488143600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.727 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:08:37.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2894783574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:37 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/488143600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2894783574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:37 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/488143600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 -- 172.21.15.6:0/3044982219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19541014e0 msgr2=0x7f1954103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3044982219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19541014e0 0x7f1954103960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1948009990 tx=0x7f1948009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 -- 172.21.15.6:0/3044982219 shutdown_connections 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3044982219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19540feb80 0x7f1954100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3044982219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1954103ea0 0x7f1954104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3044982219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19541014e0 0x7f1954103960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 -- 172.21.15.6:0/3044982219 >> 172.21.15.6:0/3044982219 conn(0x7f19540fa750 msgr2=0x7f19540fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 -- 172.21.15.6:0/3044982219 shutdown_connections 2023-11-17T00:08:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.935+0000 7f195a1ff700 1 -- 172.21.15.6:0/3044982219 wait complete. 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.936+0000 7f195a1ff700 1 Processor -- start 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 -- start start 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 0x7f195419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 0x7f195419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1954103ea0 0x7f19541a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a4180 con 0x7f1954103ea0 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a42c0 con 0x7f19540feb80 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f195a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541a4450 con 0x7f19541014e0 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f1952ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 0x7f195419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f1953fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1954103ea0 0x7f19541a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f1952ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 0x7f195419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47522/0 (socket says 172.21.15.6:47522) 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f19537fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 0x7f195419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.937+0000 7f1952ffd700 1 -- 172.21.15.6:0/3384342050 learned_addr learned my addr 172.21.15.6:0/3384342050 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 -- 172.21.15.6:0/3384342050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 msgr2=0x7f195419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 0x7f195419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 -- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1954103ea0 msgr2=0x7f19541a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1954103ea0 0x7f19541a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 -- 172.21.15.6:0/3384342050 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009c40 con 0x7f19540feb80 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f19537fe700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 0x7f195419c600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f194400e9c0 tx=0x7f194400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944004220 con 0x7f19540feb80 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480095f0 con 0x7f19540feb80 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1944004380 con 0x7f19540feb80 2023-11-17T00:08:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944003c00 con 0x7f19540feb80 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.938+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954112230 con 0x7f19540feb80 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.939+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19541123c0 con 0x7f19540feb80 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.941+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1944003d60 con 0x7f19540feb80 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.943+0000 7f1950ff9700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 0x7f193c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.943+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1944014070 con 0x7f19540feb80 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.943+0000 7f1952ffd700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 0x7f193c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.944+0000 7f1952ffd700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 0x7f193c0a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1948006010 tx=0x7f194800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:37.945+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1944096430 con 0x7f19540feb80 2023-11-17T00:08:38.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.152+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f195402cfe0 con 0x7f19540feb80 2023-11-17T00:08:38.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.153+0000 7f1950ff9700 1 -- 172.21.15.6:0/3384342050 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f194401f340 con 0x7f19540feb80 2023-11-17T00:08:38.154 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 msgr2=0x7f193c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 0x7f193c0a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1948006010 tx=0x7f194800b260 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 msgr2=0x7f195419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 0x7f195419c600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f194400e9c0 tx=0x7f194400ecd0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 shutdown_connections 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f193c0a1150 0x7f193c0a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1954103ea0 0x7f19541a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540feb80 0x7f195419c600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 --2- 172.21.15.6:0/3384342050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19541014e0 0x7f195419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 >> 172.21.15.6:0/3384342050 conn(0x7f19540fa750 msgr2=0x7f19540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 shutdown_connections 2023-11-17T00:08:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.156+0000 7f195a1ff700 1 -- 172.21.15.6:0/3384342050 wait complete. 2023-11-17T00:08:38.175 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-11-17T00:08:38.176 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:08:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 -- 172.21.15.6:0/2005051892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b0101cf0 msgr2=0x7fd2b0108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 --2- 172.21.15.6:0/2005051892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b0101cf0 0x7fd2b0108590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009f90 tx=0x7fd2a800b330 comp rx=0 tx=0).stop 2023-11-17T00:08:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 -- 172.21.15.6:0/2005051892 shutdown_connections 2023-11-17T00:08:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 --2- 172.21.15.6:0/2005051892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b0101cf0 0x7fd2b0108590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 --2- 172.21.15.6:0/2005051892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b01000b0 0x7fd2b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 --2- 172.21.15.6:0/2005051892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 0x7fd2b01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 -- 172.21.15.6:0/2005051892 >> 172.21.15.6:0/2005051892 conn(0x7fd2b00fb6b0 msgr2=0x7fd2b00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 -- 172.21.15.6:0/2005051892 shutdown_connections 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.382+0000 7fd2affff700 1 -- 172.21.15.6:0/2005051892 wait complete. 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.383+0000 7fd2affff700 1 Processor -- start 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 -- start start 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 0x7fd2b0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b0101cf0 0x7fd2b019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b019fc40 con 0x7fd2b01000b0 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b019fd80 con 0x7fd2b0101cf0 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2affff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b019ff10 con 0x7fd2b0101340 2023-11-17T00:08:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2aeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2aeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35954/0 (socket says 172.21.15.6:35954) 2023-11-17T00:08:38.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2aeffd700 1 -- 172.21.15.6:0/775693262 learned_addr learned my addr 172.21.15.6:0/775693262 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2ae7fc700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 0x7fd2b0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.384+0000 7fd2af7fe700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b0101cf0 0x7fd2b019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 -- 172.21.15.6:0/775693262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 msgr2=0x7fd2b0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 0x7fd2b0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 -- 172.21.15.6:0/775693262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b0101cf0 msgr2=0x7fd2b019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b0101cf0 0x7fd2b019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 -- 172.21.15.6:0/775693262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009520 con 0x7fd2b01000b0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2aeffd700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000eb30 tx=0x7fd2a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0009880 con 0x7fd2b01000b0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a00044e0 con 0x7fd2b01000b0 2023-11-17T00:08:38.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40095f0 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0003d80 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.385+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8009c40 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.386+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b010b910 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.388+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd2a0003f20 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.390+0000 7fd297fff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 0x7fd2980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.390+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2a0014070 con 0x7fd2b01000b0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.390+0000 7fd2ae7fc700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 0x7fd2980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.390+0000 7fd2ae7fc700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 0x7fd2980a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005fd0 tx=0x7fd2a4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:38.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.394+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd2a0096590 con 0x7fd2b01000b0 2023-11-17T00:08:38.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.652+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b010bad0 con 0x7fd2b01000b0 2023-11-17T00:08:38.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.653+0000 7fd297fff700 1 -- 172.21.15.6:0/775693262 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7fd2a0094e60 con 0x7fd2b01000b0 2023-11-17T00:08:38.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:38.654 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:08:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.655+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 msgr2=0x7fd2980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.655+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 0x7fd2980a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4005fd0 tx=0x7fd2a4009310 comp rx=0 tx=0).stop 2023-11-17T00:08:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.655+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 msgr2=0x7fd2b01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.655+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000eb30 tx=0x7fd2a000ee40 comp rx=0 tx=0).stop 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 shutdown_connections 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2980a1150 0x7fd2980a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2b01000b0 0x7fd2b01980c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b0101cf0 0x7fd2b019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 --2- 172.21.15.6:0/775693262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2b0101340 0x7fd2b0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 >> 172.21.15.6:0/775693262 conn(0x7fd2b00fb6b0 msgr2=0x7fd2b00fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 shutdown_connections 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.656+0000 7fd2affff700 1 -- 172.21.15.6:0/775693262 wait complete. 2023-11-17T00:08:38.657 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:08:38.678 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-11-17T00:08:38.678 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-11-17T00:08:38.679 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-17T00:08:38.741 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:38 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3321319028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.742 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:38 smithi006 ceph-mon[97840]: pgmap v370: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:08:38.742 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:38 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3384342050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:38 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3321319028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:38 smithi087 ceph-mon[131182]: pgmap v370: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:08:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:38 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3384342050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:38 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3321319028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:38 smithi186 ceph-mon[131487]: pgmap v370: 129 pgs: 129 active+clean; 638 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:08:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:38 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3384342050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 -- 172.21.15.6:0/1158872661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081002d0 msgr2=0x7fea081006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/1158872661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081002d0 0x7fea081006e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009990 tx=0x7fe9f8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 -- 172.21.15.6:0/1158872661 shutdown_connections 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/1158872661 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 0x7fea08108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/1158872661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081002d0 0x7fea081006e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/1158872661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081014c0 0x7fea08101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 -- 172.21.15.6:0/1158872661 >> 172.21.15.6:0/1158872661 conn(0x7fea080fb8b0 msgr2=0x7fea080fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 -- 172.21.15.6:0/1158872661 shutdown_connections 2023-11-17T00:08:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.885+0000 7fea0d9ff700 1 -- 172.21.15.6:0/1158872661 wait complete. 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 Processor -- start 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 -- start start 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081014c0 0x7fea08198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 0x7fea0819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819fe50 con 0x7fea08101e50 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819ff90 con 0x7fea081014c0 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.886+0000 7fea0d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea081a0120 con 0x7fea081002d0 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49184/0 (socket says 172.21.15.6:49184) 2023-11-17T00:08:38.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 -- 172.21.15.6:0/2332729830 learned_addr learned my addr 172.21.15.6:0/2332729830 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea077fe700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 0x7fea0819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea067fc700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081014c0 0x7fea08198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 -- 172.21.15.6:0/2332729830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081014c0 msgr2=0x7fea08198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081014c0 0x7fea08198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 -- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 msgr2=0x7fea0819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 0x7fea0819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 -- 172.21.15.6:0/2332729830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00009c40 con 0x7fea081002d0 2023-11-17T00:08:38.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.887+0000 7fea06ffd700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8004730 tx=0x7fe9f8004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.888+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801d070 con 0x7fea081002d0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.888+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80095f0 con 0x7fea081002d0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.888+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f8021410 con 0x7fea081002d0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.888+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8017650 con 0x7fea081002d0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.888+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0810bca0 con 0x7fea081002d0 2023-11-17T00:08:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.889+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0810be30 con 0x7fea081002d0 2023-11-17T00:08:38.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.891+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe9f8021810 con 0x7fea081002d0 2023-11-17T00:08:38.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.892+0000 7fea0c9fd700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 0x7fe9f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:38.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.892+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9f80ce770 con 0x7fea081002d0 2023-11-17T00:08:38.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.893+0000 7fea067fc700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 0x7fe9f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:38.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.893+0000 7fea067fc700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 0x7fe9f00a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009c60 tx=0x7fe9fc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:38.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:38.894+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe9f8097b80 con 0x7fea081002d0 2023-11-17T00:08:39.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.102+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea08060f30 con 0x7fe9f00a1150 2023-11-17T00:08:39.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.116+0000 7fea0c9fd700 1 -- 172.21.15.6:0/2332729830 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427197 (secure 0 0 0) 0x7fea08060f30 con 0x7fe9f00a1150 2023-11-17T00:08:39.123 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:39.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 msgr2=0x7fe9f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 0x7fe9f00a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009c60 tx=0x7fe9fc009260 comp rx=0 tx=0).stop 2023-11-17T00:08:39.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 msgr2=0x7fea081982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8004730 tx=0x7fe9f8004760 comp rx=0 tx=0).stop 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 shutdown_connections 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fe9f00a1150 0x7fe9f00a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea08101e50 0x7fea0819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea081002d0 0x7fea081982d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 --2- 172.21.15.6:0/2332729830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea081014c0 0x7fea08198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 >> 172.21.15.6:0/2332729830 conn(0x7fea080fb8b0 msgr2=0x7fea08106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:39.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 shutdown_connections 2023-11-17T00:08:39.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.122+0000 7fea0d9ff700 1 -- 172.21.15.6:0/2332729830 wait complete. 2023-11-17T00:08:39.130 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:08:39.145 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2023-11-17T00:08:37.778036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653399,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":385,"num_read_kb":307,"num_write":264,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82136,"kb_used_data":12152,"kb_used_omap":0,"kb_used_meta":69888,"kb_avail":1124859688,"statfs":{"total":1151940427776,"available":1151856320512,"internally_reserved":0,"allocated":12443648,"data_stored":8109424,"data_compressed":389958,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":71565312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003398"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847633+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:08:03.847633+0000","last_peered":"2023-11-17T00:08:03.847633+0000","last_clean":"2023-11-17T00:08:03.847633+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:08:03.847633+0000","last_undegraded":"2023-11-17T00:08:03.847633+0000","last_fullsized":"2023-11-17T00:08:03.847633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:21:56.490827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850194+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:08:03.850194+0000","last_peered":"2023-11-17T00:08:03.850194+0000","last_clean":"2023-11-17T00:08:03.850194+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:08:03.850194+0000","last_undegraded":"2023-11-17T00:08:03.850194+0000","last_fullsized":"2023-11-17T00:08:03.850194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:10:40.278758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849881+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:08:03.849881+0000","last_peered":"2023-11-17T00:08:03.849881+0000","last_clean":"2023-11-17T00:08:03.849881+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:08:03.849881+0000","last_undegraded":"2023-11-17T00:08:03.849881+0000","last_fullsized":"2023-11-17T00:08:03.849881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:35:49.793903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.848001+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:08:03.848001+0000","last_peered":"2023-11-17T00:08:03.848001+0000","last_clean":"2023-11-17T00:08:03.848001+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.848001+0000","last_undegraded":"2023-11-17T00:08:03.848001+0000","last_fullsized":"2023-11-17T00:08:03.848001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:15:46.371750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844787+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:08:03.844787+0000","last_peered":"2023-11-17T00:08:03.844787+0000","last_clean":"2023-11-17T00:08:03.844787+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:08:03.844787+0000","last_undegraded":"2023-11-17T00:08:03.844787+0000","last_fullsized":"2023-11-17T00:08:03.844787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:22.122959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983403+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:08:03.983403+0000","last_peered":"2023-11-17T00:08:03.983403+0000","last_clean":"2023-11-17T00:08:03.983403+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.983403+0000","last_undegraded":"2023-11-17T00:08:03.983403+0000","last_fullsized":"2023-11-17T00:08:03.983403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:57.117387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107998+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:08:04.107998+0000","last_peered":"2023-11-17T00:08:04.107998+0000","last_clean":"2023-11-17T00:08:04.107998+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:08:04.107998+0000","last_undegraded":"2023-11-17T00:08:04.107998+0000","last_fullsized":"2023-11-17T00:08:04.107998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:36:57.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213578+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:08:04.213578+0000","last_peered":"2023-11-17T00:08:04.213578+0000","last_clean":"2023-11-17T00:08:04.213578+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:08:04.213578+0000","last_undegraded":"2023-11-17T00:08:04.213578+0000","last_fullsized":"2023-11-17T00:08:04.213578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:50:48.108930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264604+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:08:04.264604+0000","last_peered":"2023-11-17T00:08:04.264604+0000","last_clean":"2023-11-17T00:08:04.264604+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:08:04.264604+0000","last_undegraded":"2023-11-17T00:08:04.264604+0000","last_fullsized":"2023-11-17T00:08:04.264604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:08:39.015361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983980+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:08:03.983980+0000","last_peered":"2023-11-17T00:08:03.983980+0000","last_clean":"2023-11-17T00:08:03.983980+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:08:03.983980+0000","last_undegraded":"2023-11-17T00:08:03.983980+0000","last_fullsized":"2023-11-17T00:08:03.983980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:17:51.571212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108637+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:08:04.108637+0000","last_peered":"2023-11-17T00:08:04.108637+0000","last_clean":"2023-11-17T00:08:04.108637+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:08:04.108637+0000","last_undegraded":"2023-11-17T00:08:04.108637+0000","last_fullsized":"2023-11-17T00:08:04.108637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:07:03.626661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847909+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:08:03.847909+0000","last_peered":"2023-11-17T00:08:03.847909+0000","last_clean":"2023-11-17T00:08:03.847909+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:08:03.847909+0000","last_undegraded":"2023-11-17T00:08:03.847909+0000","last_fullsized":"2023-11-17T00:08:03.847909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:02.516764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846547+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:08:03.846547+0000","last_peered":"2023-11-17T00:08:03.846547+0000","last_clean":"2023-11-17T00:08:03.846547+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:08:03.846547+0000","last_undegraded":"2023-11-17T00:08:03.846547+0000","last_fullsized":"2023-11-17T00:08:03.846547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:03.569332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849826+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:08:03.849826+0000","last_peered":"2023-11-17T00:08:03.849826+0000","last_clean":"2023-11-17T00:08:03.849826+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:08:03.849826+0000","last_undegraded":"2023-11-17T00:08:03.849826+0000","last_fullsized":"2023-11-17T00:08:03.849826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:25:22.625141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849335+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:08:03.849335+0000","last_peered":"2023-11-17T00:08:03.849335+0000","last_clean":"2023-11-17T00:08:03.849335+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:08:03.849335+0000","last_undegraded":"2023-11-17T00:08:03.849335+0000","last_fullsized":"2023-11-17T00:08:03.849335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:11:30.202518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263937+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:08:04.263937+0000","last_peered":"2023-11-17T00:08:04.263937+0000","last_clean":"2023-11-17T00:08:04.263937+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:04.263937+0000","last_undegraded":"2023-11-17T00:08:04.263937+0000","last_fullsized":"2023-11-17T00:08:04.263937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:35:46.189504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847628+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:08:03.847628+0000","last_peered":"2023-11-17T00:08:03.847628+0000","last_clean":"2023-11-17T00:08:03.847628+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:08:03.847628+0000","last_undegraded":"2023-11-17T00:08:03.847628+0000","last_fullsized":"2023-11-17T00:08:03.847628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:41:04.189937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852535+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:08:03.852535+0000","last_peered":"2023-11-17T00:08:03.852535+0000","last_clean":"2023-11-17T00:08:03.852535+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:08:03.852535+0000","last_undegraded":"2023-11-17T00:08:03.852535+0000","last_fullsized":"2023-11-17T00:08:03.852535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:54.543190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855381+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:08:03.855381+0000","last_peered":"2023-11-17T00:08:03.855381+0000","last_clean":"2023-11-17T00:08:03.855381+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:08:03.855381+0000","last_undegraded":"2023-11-17T00:08:03.855381+0000","last_fullsized":"2023-11-17T00:08:03.855381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:33:13.207563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855173+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:08:03.855173+0000","last_peered":"2023-11-17T00:08:03.855173+0000","last_clean":"2023-11-17T00:08:03.855173+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:08:03.855173+0000","last_undegraded":"2023-11-17T00:08:03.855173+0000","last_fullsized":"2023-11-17T00:08:03.855173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:30:14.028665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852679+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:08:03.852679+0000","last_peered":"2023-11-17T00:08:03.852679+0000","last_clean":"2023-11-17T00:08:03.852679+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:08:03.852679+0000","last_undegraded":"2023-11-17T00:08:03.852679+0000","last_fullsized":"2023-11-17T00:08:03.852679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:12:07.372586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857037+0000","last_change":"2023-11-17T00:08:03.856531+0000","last_active":"2023-11-17T00:08:03.857037+0000","last_peered":"2023-11-17T00:08:03.857037+0000","last_clean":"2023-11-17T00:08:03.857037+0000","last_became_active":"2023-11-17T00:08:03.855959+0000","last_became_peered":"2023-11-17T00:08:03.855959+0000","last_unstale":"2023-11-17T00:08:03.857037+0000","last_undegraded":"2023-11-17T00:08:03.857037+0000","last_fullsized":"2023-11-17T00:08:03.857037+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:14.852453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847214+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:08:03.847214+0000","last_peered":"2023-11-17T00:08:03.847214+0000","last_clean":"2023-11-17T00:08:03.847214+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:08:03.847214+0000","last_undegraded":"2023-11-17T00:08:03.847214+0000","last_fullsized":"2023-11-17T00:08:03.847214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:51:35.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213299+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:08:04.213299+0000","last_peered":"2023-11-17T00:08:04.213299+0000","last_clean":"2023-11-17T00:08:04.213299+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:08:04.213299+0000","last_undegraded":"2023-11-17T00:08:04.213299+0000","last_fullsized":"2023-11-17T00:08:04.213299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:08:56.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850061+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:08:03.850061+0000","last_peered":"2023-11-17T00:08:03.850061+0000","last_clean":"2023-11-17T00:08:03.850061+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:08:03.850061+0000","last_undegraded":"2023-11-17T00:08:03.850061+0000","last_fullsized":"2023-11-17T00:08:03.850061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:19:55.657467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847356+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:08:03.847356+0000","last_peered":"2023-11-17T00:08:03.847356+0000","last_clean":"2023-11-17T00:08:03.847356+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:08:03.847356+0000","last_undegraded":"2023-11-17T00:08:03.847356+0000","last_fullsized":"2023-11-17T00:08:03.847356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:40:42.046652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845271+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:08:03.845271+0000","last_peered":"2023-11-17T00:08:03.845271+0000","last_clean":"2023-11-17T00:08:03.845271+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:08:03.845271+0000","last_undegraded":"2023-11-17T00:08:03.845271+0000","last_fullsized":"2023-11-17T00:08:03.845271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:46:34.862241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264181+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:08:04.264181+0000","last_peered":"2023-11-17T00:08:04.264181+0000","last_clean":"2023-11-17T00:08:04.264181+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:08:04.264181+0000","last_undegraded":"2023-11-17T00:08:04.264181+0000","last_fullsized":"2023-11-17T00:08:04.264181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:42:55.876818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847221+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:08:03.847221+0000","last_peered":"2023-11-17T00:08:03.847221+0000","last_clean":"2023-11-17T00:08:03.847221+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:08:03.847221+0000","last_undegraded":"2023-11-17T00:08:03.847221+0000","last_fullsized":"2023-11-17T00:08:03.847221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:19:29.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845331+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:08:03.845331+0000","last_peered":"2023-11-17T00:08:03.845331+0000","last_clean":"2023-11-17T00:08:03.845331+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:08:03.845331+0000","last_undegraded":"2023-11-17T00:08:03.845331+0000","last_fullsized":"2023-11-17T00:08:03.845331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:12:06.129075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852492+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:08:03.852492+0000","last_peered":"2023-11-17T00:08:03.852492+0000","last_clean":"2023-11-17T00:08:03.852492+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:08:03.852492+0000","last_undegraded":"2023-11-17T00:08:03.852492+0000","last_fullsized":"2023-11-17T00:08:03.852492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:52.353552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854846+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:08:03.854846+0000","last_peered":"2023-11-17T00:08:03.854846+0000","last_clean":"2023-11-17T00:08:03.854846+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:08:03.854846+0000","last_undegraded":"2023-11-17T00:08:03.854846+0000","last_fullsized":"2023-11-17T00:08:03.854846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:02:25.633349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851996+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:08:03.851996+0000","last_peered":"2023-11-17T00:08:03.851996+0000","last_clean":"2023-11-17T00:08:03.851996+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:08:03.851996+0000","last_undegraded":"2023-11-17T00:08:03.851996+0000","last_fullsized":"2023-11-17T00:08:03.851996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:47.233997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844846+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:08:03.844846+0000","last_peered":"2023-11-17T00:08:03.844846+0000","last_clean":"2023-11-17T00:08:03.844846+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:08:03.844846+0000","last_undegraded":"2023-11-17T00:08:03.844846+0000","last_fullsized":"2023-11-17T00:08:03.844846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:15:29.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854671+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:08:03.854671+0000","last_peered":"2023-11-17T00:08:03.854671+0000","last_clean":"2023-11-17T00:08:03.854671+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:08:03.854671+0000","last_undegraded":"2023-11-17T00:08:03.854671+0000","last_fullsized":"2023-11-17T00:08:03.854671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:29.359449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983372+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:08:03.983372+0000","last_peered":"2023-11-17T00:08:03.983372+0000","last_clean":"2023-11-17T00:08:03.983372+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:08:03.983372+0000","last_undegraded":"2023-11-17T00:08:03.983372+0000","last_fullsized":"2023-11-17T00:08:03.983372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:58:34.049286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852202+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:08:03.852202+0000","last_peered":"2023-11-17T00:08:03.852202+0000","last_clean":"2023-11-17T00:08:03.852202+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:08:03.852202+0000","last_undegraded":"2023-11-17T00:08:03.852202+0000","last_fullsized":"2023-11-17T00:08:03.852202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847506+0000","last_change":"2023-11-17T00:07:02.907032+0000","last_active":"2023-11-17T00:08:03.847506+0000","last_peered":"2023-11-17T00:08:03.847506+0000","last_clean":"2023-11-17T00:08:03.847506+0000","last_became_active":"2023-11-17T00:07:02.906576+0000","last_became_peered":"2023-11-17T00:07:02.906576+0000","last_unstale":"2023-11-17T00:08:03.847506+0000","last_undegraded":"2023-11-17T00:08:03.847506+0000","last_fullsized":"2023-11-17T00:08:03.847506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:05:54.675471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846161+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:08:03.846161+0000","last_peered":"2023-11-17T00:08:03.846161+0000","last_clean":"2023-11-17T00:08:03.846161+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:08:03.846161+0000","last_undegraded":"2023-11-17T00:08:03.846161+0000","last_fullsized":"2023-11-17T00:08:03.846161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:03.642477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213484+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:08:04.213484+0000","last_peered":"2023-11-17T00:08:04.213484+0000","last_clean":"2023-11-17T00:08:04.213484+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:08:04.213484+0000","last_undegraded":"2023-11-17T00:08:04.213484+0000","last_fullsized":"2023-11-17T00:08:04.213484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:04.328652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850000+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:08:03.850000+0000","last_peered":"2023-11-17T00:08:03.850000+0000","last_clean":"2023-11-17T00:08:03.850000+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:08:03.850000+0000","last_undegraded":"2023-11-17T00:08:03.850000+0000","last_fullsized":"2023-11-17T00:08:03.850000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:53.311913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851536+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:08:03.851536+0000","last_peered":"2023-11-17T00:08:03.851536+0000","last_clean":"2023-11-17T00:08:03.851536+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:08:03.851536+0000","last_undegraded":"2023-11-17T00:08:03.851536+0000","last_fullsized":"2023-11-17T00:08:03.851536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:09.044455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851482+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:08:03.851482+0000","last_peered":"2023-11-17T00:08:03.851482+0000","last_clean":"2023-11-17T00:08:03.851482+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:08:03.851482+0000","last_undegraded":"2023-11-17T00:08:03.851482+0000","last_fullsized":"2023-11-17T00:08:03.851482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:46.077981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213652+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:08:04.213652+0000","last_peered":"2023-11-17T00:08:04.213652+0000","last_clean":"2023-11-17T00:08:04.213652+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:08:04.213652+0000","last_undegraded":"2023-11-17T00:08:04.213652+0000","last_fullsized":"2023-11-17T00:08:04.213652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:43:27.890099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983964+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:08:03.983964+0000","last_peered":"2023-11-17T00:08:03.983964+0000","last_clean":"2023-11-17T00:08:03.983964+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:08:03.983964+0000","last_undegraded":"2023-11-17T00:08:03.983964+0000","last_fullsized":"2023-11-17T00:08:03.983964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:28:16.131066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264508+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:08:04.264508+0000","last_peered":"2023-11-17T00:08:04.264508+0000","last_clean":"2023-11-17T00:08:04.264508+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:08:04.264508+0000","last_undegraded":"2023-11-17T00:08:04.264508+0000","last_fullsized":"2023-11-17T00:08:04.264508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:38:51.404549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851073+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:08:03.851073+0000","last_peered":"2023-11-17T00:08:03.851073+0000","last_clean":"2023-11-17T00:08:03.851073+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:08:03.851073+0000","last_undegraded":"2023-11-17T00:08:03.851073+0000","last_fullsized":"2023-11-17T00:08:03.851073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:07:36.078352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108453+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:08:04.108453+0000","last_peered":"2023-11-17T00:08:04.108453+0000","last_clean":"2023-11-17T00:08:04.108453+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:08:04.108453+0000","last_undegraded":"2023-11-17T00:08:04.108453+0000","last_fullsized":"2023-11-17T00:08:04.108453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:11:40.146505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.853641+0000","last_change":"2023-11-17T00:08:03.853641+0000","last_active":"2023-11-17T00:08:03.853641+0000","last_peered":"2023-11-17T00:08:03.853641+0000","last_clean":"2023-11-17T00:08:03.853641+0000","last_became_active":"2023-11-17T00:08:03.852982+0000","last_became_peered":"2023-11-17T00:08:03.852982+0000","last_unstale":"2023-11-17T00:08:03.853641+0000","last_undegraded":"2023-11-17T00:08:03.853641+0000","last_fullsized":"2023-11-17T00:08:03.853641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:28.723617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847803+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:08:03.847803+0000","last_peered":"2023-11-17T00:08:03.847803+0000","last_clean":"2023-11-17T00:08:03.847803+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:08:03.847803+0000","last_undegraded":"2023-11-17T00:08:03.847803+0000","last_fullsized":"2023-11-17T00:08:03.847803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:15:21.684199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852420+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:08:03.852420+0000","last_peered":"2023-11-17T00:08:03.852420+0000","last_clean":"2023-11-17T00:08:03.852420+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:08:03.852420+0000","last_undegraded":"2023-11-17T00:08:03.852420+0000","last_fullsized":"2023-11-17T00:08:03.852420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:42.759384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264242+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:08:04.264242+0000","last_peered":"2023-11-17T00:08:04.264242+0000","last_clean":"2023-11-17T00:08:04.264242+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:08:04.264242+0000","last_undegraded":"2023-11-17T00:08:04.264242+0000","last_fullsized":"2023-11-17T00:08:04.264242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:30:52.800786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851596+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:08:03.851596+0000","last_peered":"2023-11-17T00:08:03.851596+0000","last_clean":"2023-11-17T00:08:03.851596+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:08:03.851596+0000","last_undegraded":"2023-11-17T00:08:03.851596+0000","last_fullsized":"2023-11-17T00:08:03.851596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:37:04.648542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":308,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:33.027306+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:08:33.027306+0000","last_peered":"2023-11-17T00:08:33.027306+0000","last_clean":"2023-11-17T00:08:33.027306+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:08:33.027306+0000","last_undegraded":"2023-11-17T00:08:33.027306+0000","last_fullsized":"2023-11-17T00:08:33.027306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:26:09.862054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61987,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":243,"num_read_kb":182,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845080+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:08:03.845080+0000","last_peered":"2023-11-17T00:08:03.845080+0000","last_clean":"2023-11-17T00:08:03.845080+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:08:03.845080+0000","last_undegraded":"2023-11-17T00:08:03.845080+0000","last_fullsized":"2023-11-17T00:08:03.845080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:06.594615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852873+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:08:03.852873+0000","last_peered":"2023-11-17T00:08:03.852873+0000","last_clean":"2023-11-17T00:08:03.852873+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:08:03.852873+0000","last_undegraded":"2023-11-17T00:08:03.852873+0000","last_fullsized":"2023-11-17T00:08:03.852873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:19:00.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:07:03.428692+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:07:02.832461+0000","last_became_peered":"2023-11-17T00:07:02.832461+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:54:18.428663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846331+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:08:03.846331+0000","last_peered":"2023-11-17T00:08:03.846331+0000","last_clean":"2023-11-17T00:08:03.846331+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:08:03.846331+0000","last_undegraded":"2023-11-17T00:08:03.846331+0000","last_fullsized":"2023-11-17T00:08:03.846331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:33:28.090943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847175+0000","last_change":"2023-11-17T00:07:02.831797+0000","last_active":"2023-11-17T00:08:03.847175+0000","last_peered":"2023-11-17T00:08:03.847175+0000","last_clean":"2023-11-17T00:08:03.847175+0000","last_became_active":"2023-11-17T00:07:02.831339+0000","last_became_peered":"2023-11-17T00:07:02.831339+0000","last_unstale":"2023-11-17T00:08:03.847175+0000","last_undegraded":"2023-11-17T00:08:03.847175+0000","last_fullsized":"2023-11-17T00:08:03.847175+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:45.664170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852233+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:08:03.852233+0000","last_peered":"2023-11-17T00:08:03.852233+0000","last_clean":"2023-11-17T00:08:03.852233+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:08:03.852233+0000","last_undegraded":"2023-11-17T00:08:03.852233+0000","last_fullsized":"2023-11-17T00:08:03.852233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:55:14.023586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108152+0000","last_change":"2023-11-17T00:07:02.905543+0000","last_active":"2023-11-17T00:08:04.108152+0000","last_peered":"2023-11-17T00:08:04.108152+0000","last_clean":"2023-11-17T00:08:04.108152+0000","last_became_active":"2023-11-17T00:07:02.905087+0000","last_became_peered":"2023-11-17T00:07:02.905087+0000","last_unstale":"2023-11-17T00:08:04.108152+0000","last_undegraded":"2023-11-17T00:08:04.108152+0000","last_fullsized":"2023-11-17T00:08:04.108152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:54:13.888941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852088+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:08:03.852088+0000","last_peered":"2023-11-17T00:08:03.852088+0000","last_clean":"2023-11-17T00:08:03.852088+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:08:03.852088+0000","last_undegraded":"2023-11-17T00:08:03.852088+0000","last_fullsized":"2023-11-17T00:08:03.852088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:01:46.884846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847537+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:08:03.847537+0000","last_peered":"2023-11-17T00:08:03.847537+0000","last_clean":"2023-11-17T00:08:03.847537+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:08:03.847537+0000","last_undegraded":"2023-11-17T00:08:03.847537+0000","last_fullsized":"2023-11-17T00:08:03.847537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:15:45.367954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108387+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:08:04.108387+0000","last_peered":"2023-11-17T00:08:04.108387+0000","last_clean":"2023-11-17T00:08:04.108387+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:08:04.108387+0000","last_undegraded":"2023-11-17T00:08:04.108387+0000","last_fullsized":"2023-11-17T00:08:04.108387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:28.816560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846840+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:08:03.846840+0000","last_peered":"2023-11-17T00:08:03.846840+0000","last_clean":"2023-11-17T00:08:03.846840+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:08:03.846840+0000","last_undegraded":"2023-11-17T00:08:03.846840+0000","last_fullsized":"2023-11-17T00:08:03.846840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:32.403943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844650+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:08:03.844650+0000","last_peered":"2023-11-17T00:08:03.844650+0000","last_clean":"2023-11-17T00:08:03.844650+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:08:03.844650+0000","last_undegraded":"2023-11-17T00:08:03.844650+0000","last_fullsized":"2023-11-17T00:08:03.844650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:38:35.557739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851979+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:08:03.851979+0000","last_peered":"2023-11-17T00:08:03.851979+0000","last_clean":"2023-11-17T00:08:03.851979+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:08:03.851979+0000","last_undegraded":"2023-11-17T00:08:03.851979+0000","last_fullsized":"2023-11-17T00:08:03.851979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:18:47.226898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851714+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:08:03.851714+0000","last_peered":"2023-11-17T00:08:03.851714+0000","last_clean":"2023-11-17T00:08:03.851714+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:08:03.851714+0000","last_undegraded":"2023-11-17T00:08:03.851714+0000","last_fullsized":"2023-11-17T00:08:03.851714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:40:13.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845280+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:08:03.845280+0000","last_peered":"2023-11-17T00:08:03.845280+0000","last_clean":"2023-11-17T00:08:03.845280+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:08:03.845280+0000","last_undegraded":"2023-11-17T00:08:03.845280+0000","last_fullsized":"2023-11-17T00:08:03.845280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:10:28.500794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847488+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:08:03.847488+0000","last_peered":"2023-11-17T00:08:03.847488+0000","last_clean":"2023-11-17T00:08:03.847488+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:08:03.847488+0000","last_undegraded":"2023-11-17T00:08:03.847488+0000","last_fullsized":"2023-11-17T00:08:03.847488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:01:59.951206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844852+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:08:03.844852+0000","last_peered":"2023-11-17T00:08:03.844852+0000","last_clean":"2023-11-17T00:08:03.844852+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.844852+0000","last_undegraded":"2023-11-17T00:08:03.844852+0000","last_fullsized":"2023-11-17T00:08:03.844852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:20.769706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852078+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:08:03.852078+0000","last_peered":"2023-11-17T00:08:03.852078+0000","last_clean":"2023-11-17T00:08:03.852078+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:08:03.852078+0000","last_undegraded":"2023-11-17T00:08:03.852078+0000","last_fullsized":"2023-11-17T00:08:03.852078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:40:36.899273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983811+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:08:03.983811+0000","last_peered":"2023-11-17T00:08:03.983811+0000","last_clean":"2023-11-17T00:08:03.983811+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:08:03.983811+0000","last_undegraded":"2023-11-17T00:08:03.983811+0000","last_fullsized":"2023-11-17T00:08:03.983811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:53:36.530566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846570+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:08:03.846570+0000","last_peered":"2023-11-17T00:08:03.846570+0000","last_clean":"2023-11-17T00:08:03.846570+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:08:03.846570+0000","last_undegraded":"2023-11-17T00:08:03.846570+0000","last_fullsized":"2023-11-17T00:08:03.846570+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:01.945136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108107+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:08:04.108107+0000","last_peered":"2023-11-17T00:08:04.108107+0000","last_clean":"2023-11-17T00:08:04.108107+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:08:04.108107+0000","last_undegraded":"2023-11-17T00:08:04.108107+0000","last_fullsized":"2023-11-17T00:08:04.108107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:07:33.000705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983665+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:08:03.983665+0000","last_peered":"2023-11-17T00:08:03.983665+0000","last_clean":"2023-11-17T00:08:03.983665+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:08:03.983665+0000","last_undegraded":"2023-11-17T00:08:03.983665+0000","last_fullsized":"2023-11-17T00:08:03.983665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:10.497970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846842+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:08:03.846842+0000","last_peered":"2023-11-17T00:08:03.846842+0000","last_clean":"2023-11-17T00:08:03.846842+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:08:03.846842+0000","last_undegraded":"2023-11-17T00:08:03.846842+0000","last_fullsized":"2023-11-17T00:08:03.846842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:56:35.795593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851803+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:08:03.851803+0000","last_peered":"2023-11-17T00:08:03.851803+0000","last_clean":"2023-11-17T00:08:03.851803+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:08:03.851803+0000","last_undegraded":"2023-11-17T00:08:03.851803+0000","last_fullsized":"2023-11-17T00:08:03.851803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:16:58.183436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:12:41.066776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108595+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:08:04.108595+0000","last_peered":"2023-11-17T00:08:04.108595+0000","last_clean":"2023-11-17T00:08:04.108595+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:08:04.108595+0000","last_undegraded":"2023-11-17T00:08:04.108595+0000","last_fullsized":"2023-11-17T00:08:04.108595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:30.163957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213635+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:08:04.213635+0000","last_peered":"2023-11-17T00:08:04.213635+0000","last_clean":"2023-11-17T00:08:04.213635+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:08:04.213635+0000","last_undegraded":"2023-11-17T00:08:04.213635+0000","last_fullsized":"2023-11-17T00:08:04.213635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:23:03.217800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847624+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:08:03.847624+0000","last_peered":"2023-11-17T00:08:03.847624+0000","last_clean":"2023-11-17T00:08:03.847624+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:08:03.847624+0000","last_undegraded":"2023-11-17T00:08:03.847624+0000","last_fullsized":"2023-11-17T00:08:03.847624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:29:08.828256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107901+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:08:04.107901+0000","last_peered":"2023-11-17T00:08:04.107901+0000","last_clean":"2023-11-17T00:08:04.107901+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:08:04.107901+0000","last_undegraded":"2023-11-17T00:08:04.107901+0000","last_fullsized":"2023-11-17T00:08:04.107901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852528+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:08:03.852528+0000","last_peered":"2023-11-17T00:08:03.852528+0000","last_clean":"2023-11-17T00:08:03.852528+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:08:03.852528+0000","last_undegraded":"2023-11-17T00:08:03.852528+0000","last_fullsized":"2023-11-17T00:08:03.852528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:25.112450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:58:49.381194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108244+0000","last_change":"2023-11-17T00:07:02.906620+0000","last_active":"2023-11-17T00:08:04.108244+0000","last_peered":"2023-11-17T00:08:04.108244+0000","last_clean":"2023-11-17T00:08:04.108244+0000","last_became_active":"2023-11-17T00:07:02.906237+0000","last_became_peered":"2023-11-17T00:07:02.906237+0000","last_unstale":"2023-11-17T00:08:04.108244+0000","last_undegraded":"2023-11-17T00:08:04.108244+0000","last_fullsized":"2023-11-17T00:08:04.108244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:26:23.532266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847200+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:08:03.847200+0000","last_peered":"2023-11-17T00:08:03.847200+0000","last_clean":"2023-11-17T00:08:03.847200+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:08:03.847200+0000","last_undegraded":"2023-11-17T00:08:03.847200+0000","last_fullsized":"2023-11-17T00:08:03.847200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:52.246429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845164+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:08:03.845164+0000","last_peered":"2023-11-17T00:08:03.845164+0000","last_clean":"2023-11-17T00:08:03.845164+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:08:03.845164+0000","last_undegraded":"2023-11-17T00:08:03.845164+0000","last_fullsized":"2023-11-17T00:08:03.845164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:07.339417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846667+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:08:03.846667+0000","last_peered":"2023-11-17T00:08:03.846667+0000","last_clean":"2023-11-17T00:08:03.846667+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:08:03.846667+0000","last_undegraded":"2023-11-17T00:08:03.846667+0000","last_fullsized":"2023-11-17T00:08:03.846667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:53:45.282863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213467+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:08:04.213467+0000","last_peered":"2023-11-17T00:08:04.213467+0000","last_clean":"2023-11-17T00:08:04.213467+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:08:04.213467+0000","last_undegraded":"2023-11-17T00:08:04.213467+0000","last_fullsized":"2023-11-17T00:08:04.213467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:43:40.186246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:21.576683+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:08:21.576683+0000","last_peered":"2023-11-17T00:08:21.576683+0000","last_clean":"2023-11-17T00:08:21.576683+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:08:21.576683+0000","last_undegraded":"2023-11-17T00:08:21.576683+0000","last_fullsized":"2023-11-17T00:08:21.576683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:51.839878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214495+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:08:04.214495+0000","last_peered":"2023-11-17T00:08:04.214495+0000","last_clean":"2023-11-17T00:08:04.214495+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:08:04.214495+0000","last_undegraded":"2023-11-17T00:08:04.214495+0000","last_fullsized":"2023-11-17T00:08:04.214495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:35:18.102221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855170+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:08:03.855170+0000","last_peered":"2023-11-17T00:08:03.855170+0000","last_clean":"2023-11-17T00:08:03.855170+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:08:03.855170+0000","last_undegraded":"2023-11-17T00:08:03.855170+0000","last_fullsized":"2023-11-17T00:08:03.855170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:17:46.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847789+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:08:03.847789+0000","last_peered":"2023-11-17T00:08:03.847789+0000","last_clean":"2023-11-17T00:08:03.847789+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:08:03.847789+0000","last_undegraded":"2023-11-17T00:08:03.847789+0000","last_fullsized":"2023-11-17T00:08:03.847789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:30:22.305069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850179+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:08:03.850179+0000","last_peered":"2023-11-17T00:08:03.850179+0000","last_clean":"2023-11-17T00:08:03.850179+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:08:03.850179+0000","last_undegraded":"2023-11-17T00:08:03.850179+0000","last_fullsized":"2023-11-17T00:08:03.850179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:24:04.518190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847548+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:08:03.847548+0000","last_peered":"2023-11-17T00:08:03.847548+0000","last_clean":"2023-11-17T00:08:03.847548+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:08:03.847548+0000","last_undegraded":"2023-11-17T00:08:03.847548+0000","last_fullsized":"2023-11-17T00:08:03.847548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:51.426856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263432+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:08:04.263432+0000","last_peered":"2023-11-17T00:08:04.263432+0000","last_clean":"2023-11-17T00:08:04.263432+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:08:04.263432+0000","last_undegraded":"2023-11-17T00:08:04.263432+0000","last_fullsized":"2023-11-17T00:08:04.263432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:18:23.270940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844615+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:08:03.844615+0000","last_peered":"2023-11-17T00:08:03.844615+0000","last_clean":"2023-11-17T00:08:03.844615+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:08:03.844615+0000","last_undegraded":"2023-11-17T00:08:03.844615+0000","last_fullsized":"2023-11-17T00:08:03.844615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:24:09.066593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852838+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:08:03.852838+0000","last_peered":"2023-11-17T00:08:03.852838+0000","last_clean":"2023-11-17T00:08:03.852838+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:08:03.852838+0000","last_undegraded":"2023-11-17T00:08:03.852838+0000","last_fullsized":"2023-11-17T00:08:03.852838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:47.750149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213289+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:08:04.213289+0000","last_peered":"2023-11-17T00:08:04.213289+0000","last_clean":"2023-11-17T00:08:04.213289+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:08:04.213289+0000","last_undegraded":"2023-11-17T00:08:04.213289+0000","last_fullsized":"2023-11-17T00:08:04.213289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:13:03.414872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846881+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:08:03.846881+0000","last_peered":"2023-11-17T00:08:03.846881+0000","last_clean":"2023-11-17T00:08:03.846881+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:08:03.846881+0000","last_undegraded":"2023-11-17T00:08:03.846881+0000","last_fullsized":"2023-11-17T00:08:03.846881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:12:45.564310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847561+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:08:03.847561+0000","last_peered":"2023-11-17T00:08:03.847561+0000","last_clean":"2023-11-17T00:08:03.847561+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:08:03.847561+0000","last_undegraded":"2023-11-17T00:08:03.847561+0000","last_fullsized":"2023-11-17T00:08:03.847561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:02.825415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846923+0000","last_change":"2023-11-17T00:07:02.831541+0000","last_active":"2023-11-17T00:08:03.846923+0000","last_peered":"2023-11-17T00:08:03.846923+0000","last_clean":"2023-11-17T00:08:03.846923+0000","last_became_active":"2023-11-17T00:07:02.831030+0000","last_became_peered":"2023-11-17T00:07:02.831030+0000","last_unstale":"2023-11-17T00:08:03.846923+0000","last_undegraded":"2023-11-17T00:08:03.846923+0000","last_fullsized":"2023-11-17T00:08:03.846923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:41.302381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214157+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:08:04.214157+0000","last_peered":"2023-11-17T00:08:04.214157+0000","last_clean":"2023-11-17T00:08:04.214157+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:08:04.214157+0000","last_undegraded":"2023-11-17T00:08:04.214157+0000","last_fullsized":"2023-11-17T00:08:04.214157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:10:32.041314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847519+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:08:03.847519+0000","last_peered":"2023-11-17T00:08:03.847519+0000","last_clean":"2023-11-17T00:08:03.847519+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.847519+0000","last_undegraded":"2023-11-17T00:08:03.847519+0000","last_fullsized":"2023-11-17T00:08:03.847519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:38.729014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844771+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:08:03.844771+0000","last_peered":"2023-11-17T00:08:03.844771+0000","last_clean":"2023-11-17T00:08:03.844771+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:08:03.844771+0000","last_undegraded":"2023-11-17T00:08:03.844771+0000","last_fullsized":"2023-11-17T00:08:03.844771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:02.714237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108213+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:08:04.108213+0000","last_peered":"2023-11-17T00:08:04.108213+0000","last_clean":"2023-11-17T00:08:04.108213+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:08:04.108213+0000","last_undegraded":"2023-11-17T00:08:04.108213+0000","last_fullsized":"2023-11-17T00:08:04.108213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:06.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263797+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:08:04.263797+0000","last_peered":"2023-11-17T00:08:04.263797+0000","last_clean":"2023-11-17T00:08:04.263797+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:08:04.263797+0000","last_undegraded":"2023-11-17T00:08:04.263797+0000","last_fullsized":"2023-11-17T00:08:04.263797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:04:08.745867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854990+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:08:03.854990+0000","last_peered":"2023-11-17T00:08:03.854990+0000","last_clean":"2023-11-17T00:08:03.854990+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:08:03.854990+0000","last_undegraded":"2023-11-17T00:08:03.854990+0000","last_fullsized":"2023-11-17T00:08:03.854990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:47.738117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108772+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:08:04.108772+0000","last_peered":"2023-11-17T00:08:04.108772+0000","last_clean":"2023-11-17T00:08:04.108772+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:08:04.108772+0000","last_undegraded":"2023-11-17T00:08:04.108772+0000","last_fullsized":"2023-11-17T00:08:04.108772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:30:26.916829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849466+0000","last_change":"2023-11-17T00:07:02.906790+0000","last_active":"2023-11-17T00:08:03.849466+0000","last_peered":"2023-11-17T00:08:03.849466+0000","last_clean":"2023-11-17T00:08:03.849466+0000","last_became_active":"2023-11-17T00:07:02.906439+0000","last_became_peered":"2023-11-17T00:07:02.906439+0000","last_unstale":"2023-11-17T00:08:03.849466+0000","last_undegraded":"2023-11-17T00:08:03.849466+0000","last_fullsized":"2023-11-17T00:08:03.849466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:44:33.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852675+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:08:03.852675+0000","last_peered":"2023-11-17T00:08:03.852675+0000","last_clean":"2023-11-17T00:08:03.852675+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:08:03.852675+0000","last_undegraded":"2023-11-17T00:08:03.852675+0000","last_fullsized":"2023-11-17T00:08:03.852675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:09:45.189368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108257+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:08:04.108257+0000","last_peered":"2023-11-17T00:08:04.108257+0000","last_clean":"2023-11-17T00:08:04.108257+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:08:04.108257+0000","last_undegraded":"2023-11-17T00:08:04.108257+0000","last_fullsized":"2023-11-17T00:08:04.108257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:20:34.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844600+0000","last_change":"2023-11-17T00:07:02.906389+0000","last_active":"2023-11-17T00:08:03.844600+0000","last_peered":"2023-11-17T00:08:03.844600+0000","last_clean":"2023-11-17T00:08:03.844600+0000","last_became_active":"2023-11-17T00:07:02.905885+0000","last_became_peered":"2023-11-17T00:07:02.905885+0000","last_unstale":"2023-11-17T00:08:03.844600+0000","last_undegraded":"2023-11-17T00:08:03.844600+0000","last_fullsized":"2023-11-17T00:08:03.844600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:46:29.740711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846887+0000","last_change":"2023-11-17T00:07:03.431743+0000","last_active":"2023-11-17T00:08:03.846887+0000","last_peered":"2023-11-17T00:08:03.846887+0000","last_clean":"2023-11-17T00:08:03.846887+0000","last_became_active":"2023-11-17T00:07:02.907841+0000","last_became_peered":"2023-11-17T00:07:02.907841+0000","last_unstale":"2023-11-17T00:08:03.846887+0000","last_undegraded":"2023-11-17T00:08:03.846887+0000","last_fullsized":"2023-11-17T00:08:03.846887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:57:52.605327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108798+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:08:04.108798+0000","last_peered":"2023-11-17T00:08:04.108798+0000","last_clean":"2023-11-17T00:08:04.108798+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:08:04.108798+0000","last_undegraded":"2023-11-17T00:08:04.108798+0000","last_fullsized":"2023-11-17T00:08:04.108798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:57:19.682430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:52.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855451+0000","last_change":"2023-11-17T00:07:02.832453+0000","last_active":"2023-11-17T00:08:03.855451+0000","last_peered":"2023-11-17T00:08:03.855451+0000","last_clean":"2023-11-17T00:08:03.855451+0000","last_became_active":"2023-11-17T00:07:02.831920+0000","last_became_peered":"2023-11-17T00:07:02.831920+0000","last_unstale":"2023-11-17T00:08:03.855451+0000","last_undegraded":"2023-11-17T00:08:03.855451+0000","last_fullsized":"2023-11-17T00:08:03.855451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:17:01.346164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854744+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:08:03.854744+0000","last_peered":"2023-11-17T00:08:03.854744+0000","last_clean":"2023-11-17T00:08:03.854744+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:08:03.854744+0000","last_undegraded":"2023-11-17T00:08:03.854744+0000","last_fullsized":"2023-11-17T00:08:03.854744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:49:57.451212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849775+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:08:03.849775+0000","last_peered":"2023-11-17T00:08:03.849775+0000","last_clean":"2023-11-17T00:08:03.849775+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:08:03.849775+0000","last_undegraded":"2023-11-17T00:08:03.849775+0000","last_fullsized":"2023-11-17T00:08:03.849775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:19.420736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846718+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:08:03.846718+0000","last_peered":"2023-11-17T00:08:03.846718+0000","last_clean":"2023-11-17T00:08:03.846718+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:03.846718+0000","last_undegraded":"2023-11-17T00:08:03.846718+0000","last_fullsized":"2023-11-17T00:08:03.846718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:22:40.198134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857463+0000","last_change":"2023-11-17T00:08:03.856824+0000","last_active":"2023-11-17T00:08:03.857463+0000","last_peered":"2023-11-17T00:08:03.857463+0000","last_clean":"2023-11-17T00:08:03.857463+0000","last_became_active":"2023-11-17T00:08:03.856253+0000","last_became_peered":"2023-11-17T00:08:03.856253+0000","last_unstale":"2023-11-17T00:08:03.857463+0000","last_undegraded":"2023-11-17T00:08:03.857463+0000","last_fullsized":"2023-11-17T00:08:03.857463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:56:01.187155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855315+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:08:03.855315+0000","last_peered":"2023-11-17T00:08:03.855315+0000","last_clean":"2023-11-17T00:08:03.855315+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:08:03.855315+0000","last_undegraded":"2023-11-17T00:08:03.855315+0000","last_fullsized":"2023-11-17T00:08:03.855315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:24.316477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214533+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:08:04.214533+0000","last_peered":"2023-11-17T00:08:04.214533+0000","last_clean":"2023-11-17T00:08:04.214533+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:08:04.214533+0000","last_undegraded":"2023-11-17T00:08:04.214533+0000","last_fullsized":"2023-11-17T00:08:04.214533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:00:40.244245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851683+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:08:03.851683+0000","last_peered":"2023-11-17T00:08:03.851683+0000","last_clean":"2023-11-17T00:08:03.851683+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:08:03.851683+0000","last_undegraded":"2023-11-17T00:08:03.851683+0000","last_fullsized":"2023-11-17T00:08:03.851683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:02.574283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213608+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:08:04.213608+0000","last_peered":"2023-11-17T00:08:04.213608+0000","last_clean":"2023-11-17T00:08:04.213608+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:08:04.213608+0000","last_undegraded":"2023-11-17T00:08:04.213608+0000","last_fullsized":"2023-11-17T00:08:04.213608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:22:46.661532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263993+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:08:04.263993+0000","last_peered":"2023-11-17T00:08:04.263993+0000","last_clean":"2023-11-17T00:08:04.263993+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:08:04.263993+0000","last_undegraded":"2023-11-17T00:08:04.263993+0000","last_fullsized":"2023-11-17T00:08:04.263993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:43.430227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846494+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:08:03.846494+0000","last_peered":"2023-11-17T00:08:03.846494+0000","last_clean":"2023-11-17T00:08:03.846494+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:08:03.846494+0000","last_undegraded":"2023-11-17T00:08:03.846494+0000","last_fullsized":"2023-11-17T00:08:03.846494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:56.341805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847266+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:08:03.847266+0000","last_peered":"2023-11-17T00:08:03.847266+0000","last_clean":"2023-11-17T00:08:03.847266+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:08:03.847266+0000","last_undegraded":"2023-11-17T00:08:03.847266+0000","last_fullsized":"2023-11-17T00:08:03.847266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:00:02.220886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62785,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":249,"num_read_kb":188,"num_write":81,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":193734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":1007616,"data_stored":513556,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86199999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.042,"15min":1.042},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.53900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.75800000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.42199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78700000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.69199999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.495},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.54100000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.505,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432}]},{"osd":6,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.52700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.621,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91200000000000003,"5min":1.153,"15min":1.153},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.64300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.95199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.58699999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93000000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63},{"interface":"front","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78200000000000003,"15min":0.77300000000000002},"min":{"1min":0.66100000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1000000000000001,"5min":1.476,"15min":1.476},"last":0.624},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.77900000000000003},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98699999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67100000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.71099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94299999999999995,"5min":1.429,"15min":1.429},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.522,"15min":1.522},"last":0.58899999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6612,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738540,"statfs":{"total":95995035648,"available":95988264960,"internally_reserved":0,"allocated":1126400,"data_stored":1161328,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.495}]},{"osd":1,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45900000000000002,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.433,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56200000000000006,"5min":0.872,"15min":0.872},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81899999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60499999999999998,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.874,"15min":0.874},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.374}]},{"osd":5,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67900000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.48999999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84799999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81699999999999995,"5min":0.996,"15min":0.996},"last":0.45600000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.58499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.5169999999999999,"5min":3.5169999999999999,"15min":3.5169999999999999},"last":0.63}]},{"osd":11,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.61599999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.59699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61299999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6372,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738780,"statfs":{"total":95995035648,"available":95988510720,"internally_reserved":0,"allocated":1011712,"data_stored":518200,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.47499999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.622}]},{"osd":3,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.79500000000000004,"15min":0.82999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82999999999999996,"15min":0.84999999999999998},"min":{"1min":0.57099999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.175,"5min":0.91100000000000003,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.0700000000000003,"5min":6.0700000000000003,"15min":6.0700000000000003},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92500000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.94699999999999995},"min":{"1min":0.61599999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.629}]},{"osd":8,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.0189999999999999,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0519999999999996,"5min":6.0519999999999996,"15min":6.0519999999999996},"last":0.75}]},{"osd":10,"last update":"Fri Nov 17 00:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.91200000000000003,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98899999999999999,"5min":1.677,"15min":1.677},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93899999999999995,"15min":0.98499999999999999},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.77400000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.91500000000000004,"15min":0.93000000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1659999999999999,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1870000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.74199999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":514126,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98999999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.52100000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.83299999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.751}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75800000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76000000000000001,"5min":1.139,"15min":1.139},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.56100000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95599999999999996,"5min":1.075,"15min":1.075},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":1.008,"15min":1.008},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.014,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.51200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0329999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81499999999999995,"15min":0.79000000000000004},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78600000000000003,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0629999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Nov 17 00:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.72299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.125,"15min":1.125},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":1060864,"data_stored":1108568,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.317,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59399999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.433},"min":{"1min":0.375,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.627,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.70499999999999996,"5min":1.083,"15min":1.083},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.58199999999999996},"min":{"1min":0.35299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68500000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":4.7720000000000002,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66600000000000004,"15min":0.69299999999999995},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.173,"15min":1.173},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84699999999999998,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.85299999999999998}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.68300000000000005},"min":{"1min":0.58399999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.92900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.745},"min":{"1min":0.63100000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86899999999999999}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82399999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.94399999999999995}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81200000000000006,"15min":0.81100000000000005},"min":{"1min":0.70099999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.92300000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":999424,"data_stored":509426,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.5}]},{"osd":1,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.224,"15min":1.224},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70399999999999996,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92300000000000004,"5min":1.091,"15min":1.091},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61299999999999999,"15min":0.65000000000000002},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75900000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.41299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.752,"15min":0.751},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94899999999999995,"5min":1.262,"15min":1.262},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92800000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Nov 17 00:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82099999999999995,"15min":0.77000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.27,"5min":1.377,"15min":1.377},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.85399999999999998,"15min":0.85199999999999998},"min":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.89900000000000002},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.754}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.232,"5min":1.429,"15min":1.429},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94899999999999995,"15min":0.98499999999999999},"min":{"1min":0.68799999999999994,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98199999999999998,"15min":1.0109999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95699999999999996,"15min":0.98499999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.0249999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.92800000000000005,"15min":0.93200000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.0940000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0620000000000001,"5min":1.823,"15min":1.823},"last":0.78400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7652,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737500,"statfs":{"total":95995035648,"available":95987200000,"internally_reserved":0,"allocated":1077248,"data_stored":576792,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.30499999999999999,"15min":0.27100000000000002},"max":{"1min":0.86899999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.53200000000000003},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.34200000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.501,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.84399999999999997,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.55500000000000005},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66900000000000004,"15min":0.59099999999999997},"min":{"1min":0.50600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88200000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624}]},{"osd":4,"last update":"Fri Nov 17 00:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.42199999999999999},"min":{"1min":0.38400000000000001,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.67900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.42099999999999999},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":1.365,"15min":1.365},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63500000000000001,"15min":0.57299999999999995},"min":{"1min":0.57299999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88500000000000001,"5min":1.331,"15min":1.331},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.52900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":1.395,"15min":1.395},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.502,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.501,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94999999999999996,"5min":1.155,"15min":1.155},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67700000000000005,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.73499999999999999,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.747}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.58299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.423,"15min":1.423},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.52400000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.69999999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0700000000000001,"5min":1.085,"15min":1.085},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97099999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.72599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.77200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7452,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737700,"statfs":{"total":95995035648,"available":95987404800,"internally_reserved":0,"allocated":1003520,"data_stored":509516,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":1.385,"15min":1.385},"last":0.62},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68000000000000005,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.373},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0940000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.747,"15min":0.61799999999999999},"min":{"1min":0.66400000000000003,"5min":0.436,"15min":0.35399999999999998},"max":{"1min":1.1459999999999999,"5min":1.675,"15min":1.675},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.53000000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.76000000000000001,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.01}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2609999999999999,"5min":1.73,"15min":1.73},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88600000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.90800000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1299999999999999,"5min":1.917,"15min":1.917},"last":0.93200000000000005}]},{"osd":8,"last update":"Fri Nov 17 00:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.294,"5min":1.377,"15min":1.377},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.876,"15min":0.82199999999999995},"min":{"1min":0.65800000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.048}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.878,"15min":0.872},"min":{"1min":0.70399999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90400000000000003,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66700000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.149,"15min":1.232},"min":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.843,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92700000000000005,"15min":0.93700000000000006},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2230000000000001,"5min":1.377,"15min":1.377},"last":0.71499999999999997}]},{"osd":11,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.97099999999999997,"15min":0.95699999999999996},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.23,"5min":4.343,"15min":4.343},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.327,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.081}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7644,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737508,"statfs":{"total":95995035648,"available":95987208192,"internally_reserved":0,"allocated":1069056,"data_stored":571097,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.32900000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56699999999999995,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.38,"5min":0.376,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.47999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75900000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.45100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.45300000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.435,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84499999999999997,"5min":1.071,"15min":1.071},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.60099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.85099999999999998},"min":{"1min":0.66400000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64300000000000002,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94399999999999995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.56599999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.72699999999999998},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96599999999999997,"5min":1.091,"15min":1.091},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.747,"15min":0.71399999999999997},"min":{"1min":0.58799999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91100000000000003,"5min":1.01,"15min":1.01},"last":0.753},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.78900000000000003,"15min":0.75},"min":{"1min":0.627,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":1.107,"15min":1.107},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89500000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.88900000000000001,"15min":0.89500000000000002},"min":{"1min":0.79400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6812,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738340,"statfs":{"total":95995035648,"available":95988060160,"internally_reserved":0,"allocated":1003520,"data_stored":509199,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48899999999999999,"15min":0.42699999999999999},"min":{"1min":0.34799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":1.484,"15min":1.484},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.61399999999999999},"min":{"1min":0.52500000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1399999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.64700000000000002},"min":{"1min":0.46200000000000002,"5min":0.45300000000000001,"15min":0.40100000000000002},"max":{"1min":0.86499999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.90800000000000003,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73299999999999998,"15min":0.68100000000000005},"min":{"1min":0.49099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.90200000000000002,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.68400000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95899999999999996,"5min":1.359,"15min":1.359},"last":0.873}]},{"osd":7,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.76200000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.66600000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.92000000000000004},"min":{"1min":0.55200000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":1.0209999999999999},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.95599999999999996,"15min":1.0009999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4850000000000001,"5min":1.496,"15min":1.496},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.96499999999999997,"15min":1.0129999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.268,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.89100000000000001,"15min":0.90400000000000003},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":2.101,"15min":2.101},"last":0.748},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.177,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.75800000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":1007616,"data_stored":513544,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.35999999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.435,"15min":0.373},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":0.875,"15min":0.875},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.56599999999999995},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.55600000000000005},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.38300000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79600000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.626,"15min":0.57299999999999995},"min":{"1min":0.47199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67600000000000005,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88400000000000001,"5min":1,"15min":1},"last":0.37},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91200000000000003,"5min":0.999,"15min":0.999},"last":0.47099999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.105,"15min":1.105},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79000000000000004,"15min":0.80400000000000005},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":0.93799999999999994,"5min":1.081,"15min":1.081},"last":0.59899999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.441},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.39800000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82899999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.03,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.752},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6872,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738280,"statfs":{"total":95995035648,"available":95987998720,"internally_reserved":0,"allocated":1064960,"data_stored":1104072,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.62,"15min":0.56699999999999995},"min":{"1min":0.44,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.60599999999999998,"15min":0.56299999999999994},"min":{"1min":0.496,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.502}]},{"osd":2,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.436,"15min":0.39500000000000002},"min":{"1min":0.38700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.311},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.44,"15min":0.40699999999999997},"min":{"1min":0.38900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.29099999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.45300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.54100000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.434}]},{"osd":4,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60999999999999999,"15min":0.58099999999999996},"min":{"1min":0.50800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.65100000000000002,"15min":0.59099999999999997},"min":{"1min":0.51700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67600000000000005,"15min":0.66500000000000004},"min":{"1min":0.56499999999999995,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85299999999999998,"5min":1.05,"15min":1.05},"last":0.378},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84799999999999998,"5min":1.046,"15min":1.046},"last":0.46600000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.625},"min":{"1min":0.60799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.49299999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65000000000000002,"15min":0.61799999999999999},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85799999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.34300000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.60899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.022,"15min":1.022},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57699999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78700000000000003,"15min":0.77000000000000002},"min":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.073,"15min":1.073},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.92500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:08:39.171 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1285806175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 msgr2=0x7f4d8410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1285806175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8410a610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009f90 tx=0x7f4d7c00b330 comp rx=0 tx=0).stop 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1285806175 shutdown_connections 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1285806175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8410a610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1285806175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 0x7f4d84072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1285806175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d840739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1285806175 >> 172.21.15.6:0/1285806175 conn(0x7f4d840fb8b0 msgr2=0x7f4d840fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:39.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1285806175 shutdown_connections 2023-11-17T00:08:39.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.387+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1285806175 wait complete. 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 Processor -- start 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 -- start start 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d841983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 0x7f4d84198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8419ff70 con 0x7f4d84073550 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d841a00b0 con 0x7f4d84074b30 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.388+0000 7f4d8c7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d841a0240 con 0x7f4d84108190 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8a59b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d841983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d89d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 0x7f4d84198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8a59b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d841983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60432/0 (socket says 172.21.15.6:60432) 2023-11-17T00:08:39.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8a59b700 1 -- 172.21.15.6:0/1877807616 learned_addr learned my addr 172.21.15.6:0/1877807616 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 -- 172.21.15.6:0/1877807616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 msgr2=0x7f4d84198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 0x7f4d84198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 -- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 msgr2=0x7f4d841983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d841983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 -- 172.21.15.6:0/1877807616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c009c40 con 0x7f4d84108190 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.389+0000 7f4d8ad9c700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8419f9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00b6c0 tx=0x7f4d7c004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.390+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00e040 con 0x7f4d84108190 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.390+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d7c00bbf0 con 0x7f4d84108190 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.390+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8410deb0 con 0x7f4d84108190 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.390+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c005040 con 0x7f4d84108190 2023-11-17T00:08:39.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.390+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8410dd20 con 0x7f4d84108190 2023-11-17T00:08:39.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.391+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d84106840 con 0x7f4d84108190 2023-11-17T00:08:39.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.393+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4d7c01d470 con 0x7f4d84108190 2023-11-17T00:08:39.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.394+0000 7f4d7b7fe700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 0x7f4d700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.395+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4d7c0ce580 con 0x7f4d84108190 2023-11-17T00:08:39.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.395+0000 7f4d8a59b700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 0x7f4d700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.395+0000 7f4d8a59b700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 0x7f4d700a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005950 tx=0x7f4d80009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:39.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.396+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4d7c097990 con 0x7f4d84108190 2023-11-17T00:08:39.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.600+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d840619c0 con 0x7f4d700a1150 2023-11-17T00:08:39.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.614+0000 7f4d7b7fe700 1 -- 172.21.15.6:0/1877807616 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427197 (secure 0 0 0) 0x7f4d840619c0 con 0x7f4d700a1150 2023-11-17T00:08:39.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:39.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 msgr2=0x7f4d700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 0x7f4d700a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005950 tx=0x7f4d80009260 comp rx=0 tx=0).stop 2023-11-17T00:08:39.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 msgr2=0x7f4d8419f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8419f9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00b6c0 tx=0x7f4d7c004c60 comp rx=0 tx=0).stop 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 shutdown_connections 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d700a1150 0x7f4d700a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d84073550 0x7f4d841983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d84108190 0x7f4d8419f9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 --2- 172.21.15.6:0/1877807616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84074b30 0x7f4d84198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 >> 172.21.15.6:0/1877807616 conn(0x7f4d840fb8b0 msgr2=0x7f4d840fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 shutdown_connections 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.619+0000 7f4d8c7ff700 1 -- 172.21.15.6:0/1877807616 wait complete. 2023-11-17T00:08:39.625 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:08:39.639 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2023-11-17T00:08:37.778036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653399,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":385,"num_read_kb":307,"num_write":264,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82136,"kb_used_data":12152,"kb_used_omap":0,"kb_used_meta":69888,"kb_avail":1124859688,"statfs":{"total":1151940427776,"available":1151856320512,"internally_reserved":0,"allocated":12443648,"data_stored":8109424,"data_compressed":389958,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":71565312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003398"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847633+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:08:03.847633+0000","last_peered":"2023-11-17T00:08:03.847633+0000","last_clean":"2023-11-17T00:08:03.847633+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:08:03.847633+0000","last_undegraded":"2023-11-17T00:08:03.847633+0000","last_fullsized":"2023-11-17T00:08:03.847633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:21:56.490827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850194+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:08:03.850194+0000","last_peered":"2023-11-17T00:08:03.850194+0000","last_clean":"2023-11-17T00:08:03.850194+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:08:03.850194+0000","last_undegraded":"2023-11-17T00:08:03.850194+0000","last_fullsized":"2023-11-17T00:08:03.850194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:10:40.278758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849881+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:08:03.849881+0000","last_peered":"2023-11-17T00:08:03.849881+0000","last_clean":"2023-11-17T00:08:03.849881+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:08:03.849881+0000","last_undegraded":"2023-11-17T00:08:03.849881+0000","last_fullsized":"2023-11-17T00:08:03.849881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:35:49.793903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.848001+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:08:03.848001+0000","last_peered":"2023-11-17T00:08:03.848001+0000","last_clean":"2023-11-17T00:08:03.848001+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.848001+0000","last_undegraded":"2023-11-17T00:08:03.848001+0000","last_fullsized":"2023-11-17T00:08:03.848001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:15:46.371750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844787+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:08:03.844787+0000","last_peered":"2023-11-17T00:08:03.844787+0000","last_clean":"2023-11-17T00:08:03.844787+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:08:03.844787+0000","last_undegraded":"2023-11-17T00:08:03.844787+0000","last_fullsized":"2023-11-17T00:08:03.844787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:22.122959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983403+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:08:03.983403+0000","last_peered":"2023-11-17T00:08:03.983403+0000","last_clean":"2023-11-17T00:08:03.983403+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.983403+0000","last_undegraded":"2023-11-17T00:08:03.983403+0000","last_fullsized":"2023-11-17T00:08:03.983403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:57.117387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107998+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:08:04.107998+0000","last_peered":"2023-11-17T00:08:04.107998+0000","last_clean":"2023-11-17T00:08:04.107998+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:08:04.107998+0000","last_undegraded":"2023-11-17T00:08:04.107998+0000","last_fullsized":"2023-11-17T00:08:04.107998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:36:57.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213578+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:08:04.213578+0000","last_peered":"2023-11-17T00:08:04.213578+0000","last_clean":"2023-11-17T00:08:04.213578+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:08:04.213578+0000","last_undegraded":"2023-11-17T00:08:04.213578+0000","last_fullsized":"2023-11-17T00:08:04.213578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:50:48.108930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264604+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:08:04.264604+0000","last_peered":"2023-11-17T00:08:04.264604+0000","last_clean":"2023-11-17T00:08:04.264604+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:08:04.264604+0000","last_undegraded":"2023-11-17T00:08:04.264604+0000","last_fullsized":"2023-11-17T00:08:04.264604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:08:39.015361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983980+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:08:03.983980+0000","last_peered":"2023-11-17T00:08:03.983980+0000","last_clean":"2023-11-17T00:08:03.983980+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:08:03.983980+0000","last_undegraded":"2023-11-17T00:08:03.983980+0000","last_fullsized":"2023-11-17T00:08:03.983980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:17:51.571212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108637+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:08:04.108637+0000","last_peered":"2023-11-17T00:08:04.108637+0000","last_clean":"2023-11-17T00:08:04.108637+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:08:04.108637+0000","last_undegraded":"2023-11-17T00:08:04.108637+0000","last_fullsized":"2023-11-17T00:08:04.108637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:07:03.626661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847909+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:08:03.847909+0000","last_peered":"2023-11-17T00:08:03.847909+0000","last_clean":"2023-11-17T00:08:03.847909+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:08:03.847909+0000","last_undegraded":"2023-11-17T00:08:03.847909+0000","last_fullsized":"2023-11-17T00:08:03.847909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:02.516764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846547+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:08:03.846547+0000","last_peered":"2023-11-17T00:08:03.846547+0000","last_clean":"2023-11-17T00:08:03.846547+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:08:03.846547+0000","last_undegraded":"2023-11-17T00:08:03.846547+0000","last_fullsized":"2023-11-17T00:08:03.846547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:03.569332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849826+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:08:03.849826+0000","last_peered":"2023-11-17T00:08:03.849826+0000","last_clean":"2023-11-17T00:08:03.849826+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:08:03.849826+0000","last_undegraded":"2023-11-17T00:08:03.849826+0000","last_fullsized":"2023-11-17T00:08:03.849826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:25:22.625141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849335+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:08:03.849335+0000","last_peered":"2023-11-17T00:08:03.849335+0000","last_clean":"2023-11-17T00:08:03.849335+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:08:03.849335+0000","last_undegraded":"2023-11-17T00:08:03.849335+0000","last_fullsized":"2023-11-17T00:08:03.849335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:11:30.202518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263937+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:08:04.263937+0000","last_peered":"2023-11-17T00:08:04.263937+0000","last_clean":"2023-11-17T00:08:04.263937+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:04.263937+0000","last_undegraded":"2023-11-17T00:08:04.263937+0000","last_fullsized":"2023-11-17T00:08:04.263937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:35:46.189504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847628+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:08:03.847628+0000","last_peered":"2023-11-17T00:08:03.847628+0000","last_clean":"2023-11-17T00:08:03.847628+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:08:03.847628+0000","last_undegraded":"2023-11-17T00:08:03.847628+0000","last_fullsized":"2023-11-17T00:08:03.847628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:41:04.189937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852535+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:08:03.852535+0000","last_peered":"2023-11-17T00:08:03.852535+0000","last_clean":"2023-11-17T00:08:03.852535+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:08:03.852535+0000","last_undegraded":"2023-11-17T00:08:03.852535+0000","last_fullsized":"2023-11-17T00:08:03.852535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:54.543190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855381+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:08:03.855381+0000","last_peered":"2023-11-17T00:08:03.855381+0000","last_clean":"2023-11-17T00:08:03.855381+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:08:03.855381+0000","last_undegraded":"2023-11-17T00:08:03.855381+0000","last_fullsized":"2023-11-17T00:08:03.855381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:33:13.207563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855173+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:08:03.855173+0000","last_peered":"2023-11-17T00:08:03.855173+0000","last_clean":"2023-11-17T00:08:03.855173+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:08:03.855173+0000","last_undegraded":"2023-11-17T00:08:03.855173+0000","last_fullsized":"2023-11-17T00:08:03.855173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:30:14.028665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852679+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:08:03.852679+0000","last_peered":"2023-11-17T00:08:03.852679+0000","last_clean":"2023-11-17T00:08:03.852679+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:08:03.852679+0000","last_undegraded":"2023-11-17T00:08:03.852679+0000","last_fullsized":"2023-11-17T00:08:03.852679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:12:07.372586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857037+0000","last_change":"2023-11-17T00:08:03.856531+0000","last_active":"2023-11-17T00:08:03.857037+0000","last_peered":"2023-11-17T00:08:03.857037+0000","last_clean":"2023-11-17T00:08:03.857037+0000","last_became_active":"2023-11-17T00:08:03.855959+0000","last_became_peered":"2023-11-17T00:08:03.855959+0000","last_unstale":"2023-11-17T00:08:03.857037+0000","last_undegraded":"2023-11-17T00:08:03.857037+0000","last_fullsized":"2023-11-17T00:08:03.857037+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:14.852453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847214+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:08:03.847214+0000","last_peered":"2023-11-17T00:08:03.847214+0000","last_clean":"2023-11-17T00:08:03.847214+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:08:03.847214+0000","last_undegraded":"2023-11-17T00:08:03.847214+0000","last_fullsized":"2023-11-17T00:08:03.847214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:51:35.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213299+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:08:04.213299+0000","last_peered":"2023-11-17T00:08:04.213299+0000","last_clean":"2023-11-17T00:08:04.213299+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:08:04.213299+0000","last_undegraded":"2023-11-17T00:08:04.213299+0000","last_fullsized":"2023-11-17T00:08:04.213299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:08:56.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850061+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:08:03.850061+0000","last_peered":"2023-11-17T00:08:03.850061+0000","last_clean":"2023-11-17T00:08:03.850061+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:08:03.850061+0000","last_undegraded":"2023-11-17T00:08:03.850061+0000","last_fullsized":"2023-11-17T00:08:03.850061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:19:55.657467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847356+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:08:03.847356+0000","last_peered":"2023-11-17T00:08:03.847356+0000","last_clean":"2023-11-17T00:08:03.847356+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:08:03.847356+0000","last_undegraded":"2023-11-17T00:08:03.847356+0000","last_fullsized":"2023-11-17T00:08:03.847356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:40:42.046652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845271+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:08:03.845271+0000","last_peered":"2023-11-17T00:08:03.845271+0000","last_clean":"2023-11-17T00:08:03.845271+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:08:03.845271+0000","last_undegraded":"2023-11-17T00:08:03.845271+0000","last_fullsized":"2023-11-17T00:08:03.845271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:46:34.862241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264181+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:08:04.264181+0000","last_peered":"2023-11-17T00:08:04.264181+0000","last_clean":"2023-11-17T00:08:04.264181+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:08:04.264181+0000","last_undegraded":"2023-11-17T00:08:04.264181+0000","last_fullsized":"2023-11-17T00:08:04.264181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:42:55.876818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847221+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:08:03.847221+0000","last_peered":"2023-11-17T00:08:03.847221+0000","last_clean":"2023-11-17T00:08:03.847221+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:08:03.847221+0000","last_undegraded":"2023-11-17T00:08:03.847221+0000","last_fullsized":"2023-11-17T00:08:03.847221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:19:29.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845331+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:08:03.845331+0000","last_peered":"2023-11-17T00:08:03.845331+0000","last_clean":"2023-11-17T00:08:03.845331+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:08:03.845331+0000","last_undegraded":"2023-11-17T00:08:03.845331+0000","last_fullsized":"2023-11-17T00:08:03.845331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:12:06.129075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852492+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:08:03.852492+0000","last_peered":"2023-11-17T00:08:03.852492+0000","last_clean":"2023-11-17T00:08:03.852492+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:08:03.852492+0000","last_undegraded":"2023-11-17T00:08:03.852492+0000","last_fullsized":"2023-11-17T00:08:03.852492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:52.353552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854846+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:08:03.854846+0000","last_peered":"2023-11-17T00:08:03.854846+0000","last_clean":"2023-11-17T00:08:03.854846+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:08:03.854846+0000","last_undegraded":"2023-11-17T00:08:03.854846+0000","last_fullsized":"2023-11-17T00:08:03.854846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:02:25.633349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851996+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:08:03.851996+0000","last_peered":"2023-11-17T00:08:03.851996+0000","last_clean":"2023-11-17T00:08:03.851996+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:08:03.851996+0000","last_undegraded":"2023-11-17T00:08:03.851996+0000","last_fullsized":"2023-11-17T00:08:03.851996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:47.233997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844846+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:08:03.844846+0000","last_peered":"2023-11-17T00:08:03.844846+0000","last_clean":"2023-11-17T00:08:03.844846+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:08:03.844846+0000","last_undegraded":"2023-11-17T00:08:03.844846+0000","last_fullsized":"2023-11-17T00:08:03.844846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:15:29.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854671+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:08:03.854671+0000","last_peered":"2023-11-17T00:08:03.854671+0000","last_clean":"2023-11-17T00:08:03.854671+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:08:03.854671+0000","last_undegraded":"2023-11-17T00:08:03.854671+0000","last_fullsized":"2023-11-17T00:08:03.854671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:29.359449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983372+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:08:03.983372+0000","last_peered":"2023-11-17T00:08:03.983372+0000","last_clean":"2023-11-17T00:08:03.983372+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:08:03.983372+0000","last_undegraded":"2023-11-17T00:08:03.983372+0000","last_fullsized":"2023-11-17T00:08:03.983372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:58:34.049286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852202+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:08:03.852202+0000","last_peered":"2023-11-17T00:08:03.852202+0000","last_clean":"2023-11-17T00:08:03.852202+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:08:03.852202+0000","last_undegraded":"2023-11-17T00:08:03.852202+0000","last_fullsized":"2023-11-17T00:08:03.852202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847506+0000","last_change":"2023-11-17T00:07:02.907032+0000","last_active":"2023-11-17T00:08:03.847506+0000","last_peered":"2023-11-17T00:08:03.847506+0000","last_clean":"2023-11-17T00:08:03.847506+0000","last_became_active":"2023-11-17T00:07:02.906576+0000","last_became_peered":"2023-11-17T00:07:02.906576+0000","last_unstale":"2023-11-17T00:08:03.847506+0000","last_undegraded":"2023-11-17T00:08:03.847506+0000","last_fullsized":"2023-11-17T00:08:03.847506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:05:54.675471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846161+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:08:03.846161+0000","last_peered":"2023-11-17T00:08:03.846161+0000","last_clean":"2023-11-17T00:08:03.846161+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:08:03.846161+0000","last_undegraded":"2023-11-17T00:08:03.846161+0000","last_fullsized":"2023-11-17T00:08:03.846161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:03.642477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213484+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:08:04.213484+0000","last_peered":"2023-11-17T00:08:04.213484+0000","last_clean":"2023-11-17T00:08:04.213484+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:08:04.213484+0000","last_undegraded":"2023-11-17T00:08:04.213484+0000","last_fullsized":"2023-11-17T00:08:04.213484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:04.328652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850000+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:08:03.850000+0000","last_peered":"2023-11-17T00:08:03.850000+0000","last_clean":"2023-11-17T00:08:03.850000+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:08:03.850000+0000","last_undegraded":"2023-11-17T00:08:03.850000+0000","last_fullsized":"2023-11-17T00:08:03.850000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:53.311913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851536+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:08:03.851536+0000","last_peered":"2023-11-17T00:08:03.851536+0000","last_clean":"2023-11-17T00:08:03.851536+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:08:03.851536+0000","last_undegraded":"2023-11-17T00:08:03.851536+0000","last_fullsized":"2023-11-17T00:08:03.851536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:09.044455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851482+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:08:03.851482+0000","last_peered":"2023-11-17T00:08:03.851482+0000","last_clean":"2023-11-17T00:08:03.851482+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:08:03.851482+0000","last_undegraded":"2023-11-17T00:08:03.851482+0000","last_fullsized":"2023-11-17T00:08:03.851482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:46.077981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213652+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:08:04.213652+0000","last_peered":"2023-11-17T00:08:04.213652+0000","last_clean":"2023-11-17T00:08:04.213652+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:08:04.213652+0000","last_undegraded":"2023-11-17T00:08:04.213652+0000","last_fullsized":"2023-11-17T00:08:04.213652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:43:27.890099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983964+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:08:03.983964+0000","last_peered":"2023-11-17T00:08:03.983964+0000","last_clean":"2023-11-17T00:08:03.983964+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:08:03.983964+0000","last_undegraded":"2023-11-17T00:08:03.983964+0000","last_fullsized":"2023-11-17T00:08:03.983964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:28:16.131066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264508+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:08:04.264508+0000","last_peered":"2023-11-17T00:08:04.264508+0000","last_clean":"2023-11-17T00:08:04.264508+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:08:04.264508+0000","last_undegraded":"2023-11-17T00:08:04.264508+0000","last_fullsized":"2023-11-17T00:08:04.264508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:38:51.404549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851073+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:08:03.851073+0000","last_peered":"2023-11-17T00:08:03.851073+0000","last_clean":"2023-11-17T00:08:03.851073+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:08:03.851073+0000","last_undegraded":"2023-11-17T00:08:03.851073+0000","last_fullsized":"2023-11-17T00:08:03.851073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:07:36.078352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108453+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:08:04.108453+0000","last_peered":"2023-11-17T00:08:04.108453+0000","last_clean":"2023-11-17T00:08:04.108453+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:08:04.108453+0000","last_undegraded":"2023-11-17T00:08:04.108453+0000","last_fullsized":"2023-11-17T00:08:04.108453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:11:40.146505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.853641+0000","last_change":"2023-11-17T00:08:03.853641+0000","last_active":"2023-11-17T00:08:03.853641+0000","last_peered":"2023-11-17T00:08:03.853641+0000","last_clean":"2023-11-17T00:08:03.853641+0000","last_became_active":"2023-11-17T00:08:03.852982+0000","last_became_peered":"2023-11-17T00:08:03.852982+0000","last_unstale":"2023-11-17T00:08:03.853641+0000","last_undegraded":"2023-11-17T00:08:03.853641+0000","last_fullsized":"2023-11-17T00:08:03.853641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:28.723617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847803+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:08:03.847803+0000","last_peered":"2023-11-17T00:08:03.847803+0000","last_clean":"2023-11-17T00:08:03.847803+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:08:03.847803+0000","last_undegraded":"2023-11-17T00:08:03.847803+0000","last_fullsized":"2023-11-17T00:08:03.847803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:15:21.684199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852420+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:08:03.852420+0000","last_peered":"2023-11-17T00:08:03.852420+0000","last_clean":"2023-11-17T00:08:03.852420+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:08:03.852420+0000","last_undegraded":"2023-11-17T00:08:03.852420+0000","last_fullsized":"2023-11-17T00:08:03.852420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:42.759384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264242+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:08:04.264242+0000","last_peered":"2023-11-17T00:08:04.264242+0000","last_clean":"2023-11-17T00:08:04.264242+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:08:04.264242+0000","last_undegraded":"2023-11-17T00:08:04.264242+0000","last_fullsized":"2023-11-17T00:08:04.264242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:30:52.800786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851596+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:08:03.851596+0000","last_peered":"2023-11-17T00:08:03.851596+0000","last_clean":"2023-11-17T00:08:03.851596+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:08:03.851596+0000","last_undegraded":"2023-11-17T00:08:03.851596+0000","last_fullsized":"2023-11-17T00:08:03.851596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:37:04.648542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":308,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:33.027306+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:08:33.027306+0000","last_peered":"2023-11-17T00:08:33.027306+0000","last_clean":"2023-11-17T00:08:33.027306+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:08:33.027306+0000","last_undegraded":"2023-11-17T00:08:33.027306+0000","last_fullsized":"2023-11-17T00:08:33.027306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:26:09.862054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61987,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":243,"num_read_kb":182,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845080+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:08:03.845080+0000","last_peered":"2023-11-17T00:08:03.845080+0000","last_clean":"2023-11-17T00:08:03.845080+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:08:03.845080+0000","last_undegraded":"2023-11-17T00:08:03.845080+0000","last_fullsized":"2023-11-17T00:08:03.845080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:06.594615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852873+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:08:03.852873+0000","last_peered":"2023-11-17T00:08:03.852873+0000","last_clean":"2023-11-17T00:08:03.852873+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:08:03.852873+0000","last_undegraded":"2023-11-17T00:08:03.852873+0000","last_fullsized":"2023-11-17T00:08:03.852873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:19:00.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:07:03.428692+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:07:02.832461+0000","last_became_peered":"2023-11-17T00:07:02.832461+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:54:18.428663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846331+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:08:03.846331+0000","last_peered":"2023-11-17T00:08:03.846331+0000","last_clean":"2023-11-17T00:08:03.846331+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:08:03.846331+0000","last_undegraded":"2023-11-17T00:08:03.846331+0000","last_fullsized":"2023-11-17T00:08:03.846331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:33:28.090943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847175+0000","last_change":"2023-11-17T00:07:02.831797+0000","last_active":"2023-11-17T00:08:03.847175+0000","last_peered":"2023-11-17T00:08:03.847175+0000","last_clean":"2023-11-17T00:08:03.847175+0000","last_became_active":"2023-11-17T00:07:02.831339+0000","last_became_peered":"2023-11-17T00:07:02.831339+0000","last_unstale":"2023-11-17T00:08:03.847175+0000","last_undegraded":"2023-11-17T00:08:03.847175+0000","last_fullsized":"2023-11-17T00:08:03.847175+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:45.664170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852233+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:08:03.852233+0000","last_peered":"2023-11-17T00:08:03.852233+0000","last_clean":"2023-11-17T00:08:03.852233+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:08:03.852233+0000","last_undegraded":"2023-11-17T00:08:03.852233+0000","last_fullsized":"2023-11-17T00:08:03.852233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:55:14.023586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108152+0000","last_change":"2023-11-17T00:07:02.905543+0000","last_active":"2023-11-17T00:08:04.108152+0000","last_peered":"2023-11-17T00:08:04.108152+0000","last_clean":"2023-11-17T00:08:04.108152+0000","last_became_active":"2023-11-17T00:07:02.905087+0000","last_became_peered":"2023-11-17T00:07:02.905087+0000","last_unstale":"2023-11-17T00:08:04.108152+0000","last_undegraded":"2023-11-17T00:08:04.108152+0000","last_fullsized":"2023-11-17T00:08:04.108152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:54:13.888941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852088+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:08:03.852088+0000","last_peered":"2023-11-17T00:08:03.852088+0000","last_clean":"2023-11-17T00:08:03.852088+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:08:03.852088+0000","last_undegraded":"2023-11-17T00:08:03.852088+0000","last_fullsized":"2023-11-17T00:08:03.852088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:01:46.884846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847537+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:08:03.847537+0000","last_peered":"2023-11-17T00:08:03.847537+0000","last_clean":"2023-11-17T00:08:03.847537+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:08:03.847537+0000","last_undegraded":"2023-11-17T00:08:03.847537+0000","last_fullsized":"2023-11-17T00:08:03.847537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:15:45.367954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108387+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:08:04.108387+0000","last_peered":"2023-11-17T00:08:04.108387+0000","last_clean":"2023-11-17T00:08:04.108387+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:08:04.108387+0000","last_undegraded":"2023-11-17T00:08:04.108387+0000","last_fullsized":"2023-11-17T00:08:04.108387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:28.816560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846840+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:08:03.846840+0000","last_peered":"2023-11-17T00:08:03.846840+0000","last_clean":"2023-11-17T00:08:03.846840+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:08:03.846840+0000","last_undegraded":"2023-11-17T00:08:03.846840+0000","last_fullsized":"2023-11-17T00:08:03.846840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:32.403943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844650+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:08:03.844650+0000","last_peered":"2023-11-17T00:08:03.844650+0000","last_clean":"2023-11-17T00:08:03.844650+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:08:03.844650+0000","last_undegraded":"2023-11-17T00:08:03.844650+0000","last_fullsized":"2023-11-17T00:08:03.844650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:38:35.557739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851979+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:08:03.851979+0000","last_peered":"2023-11-17T00:08:03.851979+0000","last_clean":"2023-11-17T00:08:03.851979+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:08:03.851979+0000","last_undegraded":"2023-11-17T00:08:03.851979+0000","last_fullsized":"2023-11-17T00:08:03.851979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:18:47.226898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851714+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:08:03.851714+0000","last_peered":"2023-11-17T00:08:03.851714+0000","last_clean":"2023-11-17T00:08:03.851714+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:08:03.851714+0000","last_undegraded":"2023-11-17T00:08:03.851714+0000","last_fullsized":"2023-11-17T00:08:03.851714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:40:13.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845280+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:08:03.845280+0000","last_peered":"2023-11-17T00:08:03.845280+0000","last_clean":"2023-11-17T00:08:03.845280+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:08:03.845280+0000","last_undegraded":"2023-11-17T00:08:03.845280+0000","last_fullsized":"2023-11-17T00:08:03.845280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:10:28.500794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847488+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:08:03.847488+0000","last_peered":"2023-11-17T00:08:03.847488+0000","last_clean":"2023-11-17T00:08:03.847488+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:08:03.847488+0000","last_undegraded":"2023-11-17T00:08:03.847488+0000","last_fullsized":"2023-11-17T00:08:03.847488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:01:59.951206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844852+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:08:03.844852+0000","last_peered":"2023-11-17T00:08:03.844852+0000","last_clean":"2023-11-17T00:08:03.844852+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.844852+0000","last_undegraded":"2023-11-17T00:08:03.844852+0000","last_fullsized":"2023-11-17T00:08:03.844852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:20.769706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852078+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:08:03.852078+0000","last_peered":"2023-11-17T00:08:03.852078+0000","last_clean":"2023-11-17T00:08:03.852078+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:08:03.852078+0000","last_undegraded":"2023-11-17T00:08:03.852078+0000","last_fullsized":"2023-11-17T00:08:03.852078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:40:36.899273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983811+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:08:03.983811+0000","last_peered":"2023-11-17T00:08:03.983811+0000","last_clean":"2023-11-17T00:08:03.983811+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:08:03.983811+0000","last_undegraded":"2023-11-17T00:08:03.983811+0000","last_fullsized":"2023-11-17T00:08:03.983811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:53:36.530566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846570+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:08:03.846570+0000","last_peered":"2023-11-17T00:08:03.846570+0000","last_clean":"2023-11-17T00:08:03.846570+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:08:03.846570+0000","last_undegraded":"2023-11-17T00:08:03.846570+0000","last_fullsized":"2023-11-17T00:08:03.846570+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:01.945136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108107+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:08:04.108107+0000","last_peered":"2023-11-17T00:08:04.108107+0000","last_clean":"2023-11-17T00:08:04.108107+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:08:04.108107+0000","last_undegraded":"2023-11-17T00:08:04.108107+0000","last_fullsized":"2023-11-17T00:08:04.108107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:07:33.000705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983665+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:08:03.983665+0000","last_peered":"2023-11-17T00:08:03.983665+0000","last_clean":"2023-11-17T00:08:03.983665+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:08:03.983665+0000","last_undegraded":"2023-11-17T00:08:03.983665+0000","last_fullsized":"2023-11-17T00:08:03.983665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:10.497970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846842+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:08:03.846842+0000","last_peered":"2023-11-17T00:08:03.846842+0000","last_clean":"2023-11-17T00:08:03.846842+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:08:03.846842+0000","last_undegraded":"2023-11-17T00:08:03.846842+0000","last_fullsized":"2023-11-17T00:08:03.846842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:56:35.795593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851803+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:08:03.851803+0000","last_peered":"2023-11-17T00:08:03.851803+0000","last_clean":"2023-11-17T00:08:03.851803+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:08:03.851803+0000","last_undegraded":"2023-11-17T00:08:03.851803+0000","last_fullsized":"2023-11-17T00:08:03.851803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:16:58.183436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:12:41.066776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108595+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:08:04.108595+0000","last_peered":"2023-11-17T00:08:04.108595+0000","last_clean":"2023-11-17T00:08:04.108595+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:08:04.108595+0000","last_undegraded":"2023-11-17T00:08:04.108595+0000","last_fullsized":"2023-11-17T00:08:04.108595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:30.163957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213635+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:08:04.213635+0000","last_peered":"2023-11-17T00:08:04.213635+0000","last_clean":"2023-11-17T00:08:04.213635+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:08:04.213635+0000","last_undegraded":"2023-11-17T00:08:04.213635+0000","last_fullsized":"2023-11-17T00:08:04.213635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:23:03.217800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847624+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:08:03.847624+0000","last_peered":"2023-11-17T00:08:03.847624+0000","last_clean":"2023-11-17T00:08:03.847624+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:08:03.847624+0000","last_undegraded":"2023-11-17T00:08:03.847624+0000","last_fullsized":"2023-11-17T00:08:03.847624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:29:08.828256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107901+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:08:04.107901+0000","last_peered":"2023-11-17T00:08:04.107901+0000","last_clean":"2023-11-17T00:08:04.107901+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:08:04.107901+0000","last_undegraded":"2023-11-17T00:08:04.107901+0000","last_fullsized":"2023-11-17T00:08:04.107901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852528+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:08:03.852528+0000","last_peered":"2023-11-17T00:08:03.852528+0000","last_clean":"2023-11-17T00:08:03.852528+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:08:03.852528+0000","last_undegraded":"2023-11-17T00:08:03.852528+0000","last_fullsized":"2023-11-17T00:08:03.852528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:25.112450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:58:49.381194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108244+0000","last_change":"2023-11-17T00:07:02.906620+0000","last_active":"2023-11-17T00:08:04.108244+0000","last_peered":"2023-11-17T00:08:04.108244+0000","last_clean":"2023-11-17T00:08:04.108244+0000","last_became_active":"2023-11-17T00:07:02.906237+0000","last_became_peered":"2023-11-17T00:07:02.906237+0000","last_unstale":"2023-11-17T00:08:04.108244+0000","last_undegraded":"2023-11-17T00:08:04.108244+0000","last_fullsized":"2023-11-17T00:08:04.108244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:26:23.532266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847200+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:08:03.847200+0000","last_peered":"2023-11-17T00:08:03.847200+0000","last_clean":"2023-11-17T00:08:03.847200+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:08:03.847200+0000","last_undegraded":"2023-11-17T00:08:03.847200+0000","last_fullsized":"2023-11-17T00:08:03.847200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:52.246429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845164+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:08:03.845164+0000","last_peered":"2023-11-17T00:08:03.845164+0000","last_clean":"2023-11-17T00:08:03.845164+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:08:03.845164+0000","last_undegraded":"2023-11-17T00:08:03.845164+0000","last_fullsized":"2023-11-17T00:08:03.845164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:07.339417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846667+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:08:03.846667+0000","last_peered":"2023-11-17T00:08:03.846667+0000","last_clean":"2023-11-17T00:08:03.846667+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:08:03.846667+0000","last_undegraded":"2023-11-17T00:08:03.846667+0000","last_fullsized":"2023-11-17T00:08:03.846667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:53:45.282863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213467+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:08:04.213467+0000","last_peered":"2023-11-17T00:08:04.213467+0000","last_clean":"2023-11-17T00:08:04.213467+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:08:04.213467+0000","last_undegraded":"2023-11-17T00:08:04.213467+0000","last_fullsized":"2023-11-17T00:08:04.213467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:43:40.186246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:21.576683+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:08:21.576683+0000","last_peered":"2023-11-17T00:08:21.576683+0000","last_clean":"2023-11-17T00:08:21.576683+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:08:21.576683+0000","last_undegraded":"2023-11-17T00:08:21.576683+0000","last_fullsized":"2023-11-17T00:08:21.576683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:51.839878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214495+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:08:04.214495+0000","last_peered":"2023-11-17T00:08:04.214495+0000","last_clean":"2023-11-17T00:08:04.214495+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:08:04.214495+0000","last_undegraded":"2023-11-17T00:08:04.214495+0000","last_fullsized":"2023-11-17T00:08:04.214495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:35:18.102221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855170+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:08:03.855170+0000","last_peered":"2023-11-17T00:08:03.855170+0000","last_clean":"2023-11-17T00:08:03.855170+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:08:03.855170+0000","last_undegraded":"2023-11-17T00:08:03.855170+0000","last_fullsized":"2023-11-17T00:08:03.855170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:17:46.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847789+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:08:03.847789+0000","last_peered":"2023-11-17T00:08:03.847789+0000","last_clean":"2023-11-17T00:08:03.847789+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:08:03.847789+0000","last_undegraded":"2023-11-17T00:08:03.847789+0000","last_fullsized":"2023-11-17T00:08:03.847789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:30:22.305069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850179+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:08:03.850179+0000","last_peered":"2023-11-17T00:08:03.850179+0000","last_clean":"2023-11-17T00:08:03.850179+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:08:03.850179+0000","last_undegraded":"2023-11-17T00:08:03.850179+0000","last_fullsized":"2023-11-17T00:08:03.850179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:24:04.518190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847548+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:08:03.847548+0000","last_peered":"2023-11-17T00:08:03.847548+0000","last_clean":"2023-11-17T00:08:03.847548+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:08:03.847548+0000","last_undegraded":"2023-11-17T00:08:03.847548+0000","last_fullsized":"2023-11-17T00:08:03.847548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:51.426856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263432+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:08:04.263432+0000","last_peered":"2023-11-17T00:08:04.263432+0000","last_clean":"2023-11-17T00:08:04.263432+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:08:04.263432+0000","last_undegraded":"2023-11-17T00:08:04.263432+0000","last_fullsized":"2023-11-17T00:08:04.263432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:18:23.270940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844615+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:08:03.844615+0000","last_peered":"2023-11-17T00:08:03.844615+0000","last_clean":"2023-11-17T00:08:03.844615+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:08:03.844615+0000","last_undegraded":"2023-11-17T00:08:03.844615+0000","last_fullsized":"2023-11-17T00:08:03.844615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:24:09.066593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852838+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:08:03.852838+0000","last_peered":"2023-11-17T00:08:03.852838+0000","last_clean":"2023-11-17T00:08:03.852838+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:08:03.852838+0000","last_undegraded":"2023-11-17T00:08:03.852838+0000","last_fullsized":"2023-11-17T00:08:03.852838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:47.750149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213289+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:08:04.213289+0000","last_peered":"2023-11-17T00:08:04.213289+0000","last_clean":"2023-11-17T00:08:04.213289+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:08:04.213289+0000","last_undegraded":"2023-11-17T00:08:04.213289+0000","last_fullsized":"2023-11-17T00:08:04.213289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:13:03.414872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846881+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:08:03.846881+0000","last_peered":"2023-11-17T00:08:03.846881+0000","last_clean":"2023-11-17T00:08:03.846881+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:08:03.846881+0000","last_undegraded":"2023-11-17T00:08:03.846881+0000","last_fullsized":"2023-11-17T00:08:03.846881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:12:45.564310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847561+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:08:03.847561+0000","last_peered":"2023-11-17T00:08:03.847561+0000","last_clean":"2023-11-17T00:08:03.847561+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:08:03.847561+0000","last_undegraded":"2023-11-17T00:08:03.847561+0000","last_fullsized":"2023-11-17T00:08:03.847561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:02.825415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846923+0000","last_change":"2023-11-17T00:07:02.831541+0000","last_active":"2023-11-17T00:08:03.846923+0000","last_peered":"2023-11-17T00:08:03.846923+0000","last_clean":"2023-11-17T00:08:03.846923+0000","last_became_active":"2023-11-17T00:07:02.831030+0000","last_became_peered":"2023-11-17T00:07:02.831030+0000","last_unstale":"2023-11-17T00:08:03.846923+0000","last_undegraded":"2023-11-17T00:08:03.846923+0000","last_fullsized":"2023-11-17T00:08:03.846923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:41.302381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214157+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:08:04.214157+0000","last_peered":"2023-11-17T00:08:04.214157+0000","last_clean":"2023-11-17T00:08:04.214157+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:08:04.214157+0000","last_undegraded":"2023-11-17T00:08:04.214157+0000","last_fullsized":"2023-11-17T00:08:04.214157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:10:32.041314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847519+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:08:03.847519+0000","last_peered":"2023-11-17T00:08:03.847519+0000","last_clean":"2023-11-17T00:08:03.847519+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.847519+0000","last_undegraded":"2023-11-17T00:08:03.847519+0000","last_fullsized":"2023-11-17T00:08:03.847519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:38.729014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844771+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:08:03.844771+0000","last_peered":"2023-11-17T00:08:03.844771+0000","last_clean":"2023-11-17T00:08:03.844771+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:08:03.844771+0000","last_undegraded":"2023-11-17T00:08:03.844771+0000","last_fullsized":"2023-11-17T00:08:03.844771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:02.714237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108213+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:08:04.108213+0000","last_peered":"2023-11-17T00:08:04.108213+0000","last_clean":"2023-11-17T00:08:04.108213+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:08:04.108213+0000","last_undegraded":"2023-11-17T00:08:04.108213+0000","last_fullsized":"2023-11-17T00:08:04.108213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:06.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263797+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:08:04.263797+0000","last_peered":"2023-11-17T00:08:04.263797+0000","last_clean":"2023-11-17T00:08:04.263797+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:08:04.263797+0000","last_undegraded":"2023-11-17T00:08:04.263797+0000","last_fullsized":"2023-11-17T00:08:04.263797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:04:08.745867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854990+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:08:03.854990+0000","last_peered":"2023-11-17T00:08:03.854990+0000","last_clean":"2023-11-17T00:08:03.854990+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:08:03.854990+0000","last_undegraded":"2023-11-17T00:08:03.854990+0000","last_fullsized":"2023-11-17T00:08:03.854990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:47.738117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108772+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:08:04.108772+0000","last_peered":"2023-11-17T00:08:04.108772+0000","last_clean":"2023-11-17T00:08:04.108772+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:08:04.108772+0000","last_undegraded":"2023-11-17T00:08:04.108772+0000","last_fullsized":"2023-11-17T00:08:04.108772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:30:26.916829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849466+0000","last_change":"2023-11-17T00:07:02.906790+0000","last_active":"2023-11-17T00:08:03.849466+0000","last_peered":"2023-11-17T00:08:03.849466+0000","last_clean":"2023-11-17T00:08:03.849466+0000","last_became_active":"2023-11-17T00:07:02.906439+0000","last_became_peered":"2023-11-17T00:07:02.906439+0000","last_unstale":"2023-11-17T00:08:03.849466+0000","last_undegraded":"2023-11-17T00:08:03.849466+0000","last_fullsized":"2023-11-17T00:08:03.849466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:44:33.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852675+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:08:03.852675+0000","last_peered":"2023-11-17T00:08:03.852675+0000","last_clean":"2023-11-17T00:08:03.852675+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:08:03.852675+0000","last_undegraded":"2023-11-17T00:08:03.852675+0000","last_fullsized":"2023-11-17T00:08:03.852675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:09:45.189368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108257+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:08:04.108257+0000","last_peered":"2023-11-17T00:08:04.108257+0000","last_clean":"2023-11-17T00:08:04.108257+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:08:04.108257+0000","last_undegraded":"2023-11-17T00:08:04.108257+0000","last_fullsized":"2023-11-17T00:08:04.108257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:20:34.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844600+0000","last_change":"2023-11-17T00:07:02.906389+0000","last_active":"2023-11-17T00:08:03.844600+0000","last_peered":"2023-11-17T00:08:03.844600+0000","last_clean":"2023-11-17T00:08:03.844600+0000","last_became_active":"2023-11-17T00:07:02.905885+0000","last_became_peered":"2023-11-17T00:07:02.905885+0000","last_unstale":"2023-11-17T00:08:03.844600+0000","last_undegraded":"2023-11-17T00:08:03.844600+0000","last_fullsized":"2023-11-17T00:08:03.844600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:46:29.740711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846887+0000","last_change":"2023-11-17T00:07:03.431743+0000","last_active":"2023-11-17T00:08:03.846887+0000","last_peered":"2023-11-17T00:08:03.846887+0000","last_clean":"2023-11-17T00:08:03.846887+0000","last_became_active":"2023-11-17T00:07:02.907841+0000","last_became_peered":"2023-11-17T00:07:02.907841+0000","last_unstale":"2023-11-17T00:08:03.846887+0000","last_undegraded":"2023-11-17T00:08:03.846887+0000","last_fullsized":"2023-11-17T00:08:03.846887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:57:52.605327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108798+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:08:04.108798+0000","last_peered":"2023-11-17T00:08:04.108798+0000","last_clean":"2023-11-17T00:08:04.108798+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:08:04.108798+0000","last_undegraded":"2023-11-17T00:08:04.108798+0000","last_fullsized":"2023-11-17T00:08:04.108798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:57:19.682430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:52.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855451+0000","last_change":"2023-11-17T00:07:02.832453+0000","last_active":"2023-11-17T00:08:03.855451+0000","last_peered":"2023-11-17T00:08:03.855451+0000","last_clean":"2023-11-17T00:08:03.855451+0000","last_became_active":"2023-11-17T00:07:02.831920+0000","last_became_peered":"2023-11-17T00:07:02.831920+0000","last_unstale":"2023-11-17T00:08:03.855451+0000","last_undegraded":"2023-11-17T00:08:03.855451+0000","last_fullsized":"2023-11-17T00:08:03.855451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:17:01.346164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854744+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:08:03.854744+0000","last_peered":"2023-11-17T00:08:03.854744+0000","last_clean":"2023-11-17T00:08:03.854744+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:08:03.854744+0000","last_undegraded":"2023-11-17T00:08:03.854744+0000","last_fullsized":"2023-11-17T00:08:03.854744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:49:57.451212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849775+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:08:03.849775+0000","last_peered":"2023-11-17T00:08:03.849775+0000","last_clean":"2023-11-17T00:08:03.849775+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:08:03.849775+0000","last_undegraded":"2023-11-17T00:08:03.849775+0000","last_fullsized":"2023-11-17T00:08:03.849775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:19.420736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846718+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:08:03.846718+0000","last_peered":"2023-11-17T00:08:03.846718+0000","last_clean":"2023-11-17T00:08:03.846718+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:03.846718+0000","last_undegraded":"2023-11-17T00:08:03.846718+0000","last_fullsized":"2023-11-17T00:08:03.846718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:22:40.198134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857463+0000","last_change":"2023-11-17T00:08:03.856824+0000","last_active":"2023-11-17T00:08:03.857463+0000","last_peered":"2023-11-17T00:08:03.857463+0000","last_clean":"2023-11-17T00:08:03.857463+0000","last_became_active":"2023-11-17T00:08:03.856253+0000","last_became_peered":"2023-11-17T00:08:03.856253+0000","last_unstale":"2023-11-17T00:08:03.857463+0000","last_undegraded":"2023-11-17T00:08:03.857463+0000","last_fullsized":"2023-11-17T00:08:03.857463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:56:01.187155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855315+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:08:03.855315+0000","last_peered":"2023-11-17T00:08:03.855315+0000","last_clean":"2023-11-17T00:08:03.855315+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:08:03.855315+0000","last_undegraded":"2023-11-17T00:08:03.855315+0000","last_fullsized":"2023-11-17T00:08:03.855315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:24.316477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214533+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:08:04.214533+0000","last_peered":"2023-11-17T00:08:04.214533+0000","last_clean":"2023-11-17T00:08:04.214533+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:08:04.214533+0000","last_undegraded":"2023-11-17T00:08:04.214533+0000","last_fullsized":"2023-11-17T00:08:04.214533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:00:40.244245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851683+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:08:03.851683+0000","last_peered":"2023-11-17T00:08:03.851683+0000","last_clean":"2023-11-17T00:08:03.851683+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:08:03.851683+0000","last_undegraded":"2023-11-17T00:08:03.851683+0000","last_fullsized":"2023-11-17T00:08:03.851683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:02.574283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213608+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:08:04.213608+0000","last_peered":"2023-11-17T00:08:04.213608+0000","last_clean":"2023-11-17T00:08:04.213608+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:08:04.213608+0000","last_undegraded":"2023-11-17T00:08:04.213608+0000","last_fullsized":"2023-11-17T00:08:04.213608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:22:46.661532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263993+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:08:04.263993+0000","last_peered":"2023-11-17T00:08:04.263993+0000","last_clean":"2023-11-17T00:08:04.263993+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:08:04.263993+0000","last_undegraded":"2023-11-17T00:08:04.263993+0000","last_fullsized":"2023-11-17T00:08:04.263993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:43.430227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846494+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:08:03.846494+0000","last_peered":"2023-11-17T00:08:03.846494+0000","last_clean":"2023-11-17T00:08:03.846494+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:08:03.846494+0000","last_undegraded":"2023-11-17T00:08:03.846494+0000","last_fullsized":"2023-11-17T00:08:03.846494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:56.341805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847266+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:08:03.847266+0000","last_peered":"2023-11-17T00:08:03.847266+0000","last_clean":"2023-11-17T00:08:03.847266+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:08:03.847266+0000","last_undegraded":"2023-11-17T00:08:03.847266+0000","last_fullsized":"2023-11-17T00:08:03.847266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:00:02.220886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62785,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":249,"num_read_kb":188,"num_write":81,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":193734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":1007616,"data_stored":513556,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86199999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.042,"15min":1.042},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.53900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.75800000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.42199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78700000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.69199999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.495},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.54100000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.505,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432}]},{"osd":6,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.52700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.621,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91200000000000003,"5min":1.153,"15min":1.153},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.64300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.95199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.58699999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93000000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63},{"interface":"front","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78200000000000003,"15min":0.77300000000000002},"min":{"1min":0.66100000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1000000000000001,"5min":1.476,"15min":1.476},"last":0.624},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.77900000000000003},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98699999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67100000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.71099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94299999999999995,"5min":1.429,"15min":1.429},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.522,"15min":1.522},"last":0.58899999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678062,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6612,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738540,"statfs":{"total":95995035648,"available":95988264960,"internally_reserved":0,"allocated":1126400,"data_stored":1161328,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.495}]},{"osd":1,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45900000000000002,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.433,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56200000000000006,"5min":0.872,"15min":0.872},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81899999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60499999999999998,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.874,"15min":0.874},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.374}]},{"osd":5,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67900000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.48999999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84799999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81699999999999995,"5min":0.996,"15min":0.996},"last":0.45600000000000002}]},{"osd":9,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.58499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.5169999999999999,"5min":3.5169999999999999,"15min":3.5169999999999999},"last":0.63}]},{"osd":11,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.61599999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.59699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61299999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6372,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738780,"statfs":{"total":95995035648,"available":95988510720,"internally_reserved":0,"allocated":1011712,"data_stored":518200,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.47499999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.622}]},{"osd":3,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.79500000000000004,"15min":0.82999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82999999999999996,"15min":0.84999999999999998},"min":{"1min":0.57099999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.175,"5min":0.91100000000000003,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.0700000000000003,"5min":6.0700000000000003,"15min":6.0700000000000003},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92500000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.94699999999999995},"min":{"1min":0.61599999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.629}]},{"osd":8,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.0189999999999999,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0519999999999996,"5min":6.0519999999999996,"15min":6.0519999999999996},"last":0.75}]},{"osd":10,"last update":"Fri Nov 17 00:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.91200000000000003,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98899999999999999,"5min":1.677,"15min":1.677},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93899999999999995,"15min":0.98499999999999999},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.77400000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.91500000000000004,"15min":0.93000000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1659999999999999,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1870000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.74199999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":514126,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98999999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.52100000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.83299999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.751}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75800000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76000000000000001,"5min":1.139,"15min":1.139},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.56100000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95599999999999996,"5min":1.075,"15min":1.075},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":1.008,"15min":1.008},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.014,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.51200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0329999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81499999999999995,"15min":0.79000000000000004},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78600000000000003,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0629999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Nov 17 00:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.72299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.125,"15min":1.125},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":1060864,"data_stored":1108568,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.317,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59399999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.433},"min":{"1min":0.375,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.627,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.70499999999999996,"5min":1.083,"15min":1.083},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.58199999999999996},"min":{"1min":0.35299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68500000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":4.7720000000000002,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66600000000000004,"15min":0.69299999999999995},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.173,"15min":1.173},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84699999999999998,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.85299999999999998}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.68300000000000005},"min":{"1min":0.58399999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.92900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.745},"min":{"1min":0.63100000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86899999999999999}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82399999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.94399999999999995}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81200000000000006,"15min":0.81100000000000005},"min":{"1min":0.70099999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.92300000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":999424,"data_stored":509426,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.5}]},{"osd":1,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.224,"15min":1.224},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70399999999999996,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92300000000000004,"5min":1.091,"15min":1.091},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61299999999999999,"15min":0.65000000000000002},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75900000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.41299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.752,"15min":0.751},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94899999999999995,"5min":1.262,"15min":1.262},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92800000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Nov 17 00:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82099999999999995,"15min":0.77000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.27,"5min":1.377,"15min":1.377},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.85399999999999998,"15min":0.85199999999999998},"min":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.89900000000000002},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.754}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.232,"5min":1.429,"15min":1.429},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94899999999999995,"15min":0.98499999999999999},"min":{"1min":0.68799999999999994,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98199999999999998,"15min":1.0109999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95699999999999996,"15min":0.98499999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.0249999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.92800000000000005,"15min":0.93200000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.0940000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0620000000000001,"5min":1.823,"15min":1.823},"last":0.78400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7652,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737500,"statfs":{"total":95995035648,"available":95987200000,"internally_reserved":0,"allocated":1077248,"data_stored":576792,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.30499999999999999,"15min":0.27100000000000002},"max":{"1min":0.86899999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.53200000000000003},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.34200000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.501,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.84399999999999997,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.55500000000000005},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66900000000000004,"15min":0.59099999999999997},"min":{"1min":0.50600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88200000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624}]},{"osd":4,"last update":"Fri Nov 17 00:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.42199999999999999},"min":{"1min":0.38400000000000001,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.67900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.42099999999999999},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":1.365,"15min":1.365},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63500000000000001,"15min":0.57299999999999995},"min":{"1min":0.57299999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88500000000000001,"5min":1.331,"15min":1.331},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.52900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":1.395,"15min":1.395},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.502,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.501,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94999999999999996,"5min":1.155,"15min":1.155},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67700000000000005,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.73499999999999999,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.747}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.58299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.423,"15min":1.423},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.52400000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.69999999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0700000000000001,"5min":1.085,"15min":1.085},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97099999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.72599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.77200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7452,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737700,"statfs":{"total":95995035648,"available":95987404800,"internally_reserved":0,"allocated":1003520,"data_stored":509516,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":1.385,"15min":1.385},"last":0.62},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68000000000000005,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.373},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0940000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.747,"15min":0.61799999999999999},"min":{"1min":0.66400000000000003,"5min":0.436,"15min":0.35399999999999998},"max":{"1min":1.1459999999999999,"5min":1.675,"15min":1.675},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.53000000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.76000000000000001,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.01}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2609999999999999,"5min":1.73,"15min":1.73},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88600000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.90800000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1299999999999999,"5min":1.917,"15min":1.917},"last":0.93200000000000005}]},{"osd":8,"last update":"Fri Nov 17 00:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.294,"5min":1.377,"15min":1.377},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.876,"15min":0.82199999999999995},"min":{"1min":0.65800000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.048}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.878,"15min":0.872},"min":{"1min":0.70399999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90400000000000003,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66700000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.149,"15min":1.232},"min":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.843,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92700000000000005,"15min":0.93700000000000006},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2230000000000001,"5min":1.377,"15min":1.377},"last":0.71499999999999997}]},{"osd":11,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.97099999999999997,"15min":0.95699999999999996},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.23,"5min":4.343,"15min":4.343},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.327,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.081}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7644,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737508,"statfs":{"total":95995035648,"available":95987208192,"internally_reserved":0,"allocated":1069056,"data_stored":571097,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.32900000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56699999999999995,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.38,"5min":0.376,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.47999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75900000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.45100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.45300000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.435,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84499999999999997,"5min":1.071,"15min":1.071},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.60099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.85099999999999998},"min":{"1min":0.66400000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64300000000000002,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94399999999999995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.56599999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.72699999999999998},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96599999999999997,"5min":1.091,"15min":1.091},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.747,"15min":0.71399999999999997},"min":{"1min":0.58799999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91100000000000003,"5min":1.01,"15min":1.01},"last":0.753},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.78900000000000003,"15min":0.75},"min":{"1min":0.627,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":1.107,"15min":1.107},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89500000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.88900000000000001,"15min":0.89500000000000002},"min":{"1min":0.79400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6812,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738340,"statfs":{"total":95995035648,"available":95988060160,"internally_reserved":0,"allocated":1003520,"data_stored":509199,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48899999999999999,"15min":0.42699999999999999},"min":{"1min":0.34799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":1.484,"15min":1.484},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.61399999999999999},"min":{"1min":0.52500000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1399999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.64700000000000002},"min":{"1min":0.46200000000000002,"5min":0.45300000000000001,"15min":0.40100000000000002},"max":{"1min":0.86499999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.90800000000000003,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73299999999999998,"15min":0.68100000000000005},"min":{"1min":0.49099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.90200000000000002,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.68400000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95899999999999996,"5min":1.359,"15min":1.359},"last":0.873}]},{"osd":7,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.76200000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.66600000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.92000000000000004},"min":{"1min":0.55200000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":1.0209999999999999},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.95599999999999996,"15min":1.0009999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4850000000000001,"5min":1.496,"15min":1.496},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.96499999999999997,"15min":1.0129999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.268,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.89100000000000001,"15min":0.90400000000000003},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":2.101,"15min":2.101},"last":0.748},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.177,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.75800000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":1007616,"data_stored":513544,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.35999999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.435,"15min":0.373},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":0.875,"15min":0.875},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.56599999999999995},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.55600000000000005},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.38300000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79600000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.626,"15min":0.57299999999999995},"min":{"1min":0.47199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67600000000000005,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88400000000000001,"5min":1,"15min":1},"last":0.37},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91200000000000003,"5min":0.999,"15min":0.999},"last":0.47099999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.105,"15min":1.105},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79000000000000004,"15min":0.80400000000000005},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":0.93799999999999994,"5min":1.081,"15min":1.081},"last":0.59899999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.441},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.39800000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82899999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.03,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.752},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6872,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738280,"statfs":{"total":95995035648,"available":95987998720,"internally_reserved":0,"allocated":1064960,"data_stored":1104072,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.62,"15min":0.56699999999999995},"min":{"1min":0.44,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.60599999999999998,"15min":0.56299999999999994},"min":{"1min":0.496,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.502}]},{"osd":2,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.436,"15min":0.39500000000000002},"min":{"1min":0.38700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.311},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.44,"15min":0.40699999999999997},"min":{"1min":0.38900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.29099999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.45300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.54100000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.434}]},{"osd":4,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60999999999999999,"15min":0.58099999999999996},"min":{"1min":0.50800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.65100000000000002,"15min":0.59099999999999997},"min":{"1min":0.51700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67600000000000005,"15min":0.66500000000000004},"min":{"1min":0.56499999999999995,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85299999999999998,"5min":1.05,"15min":1.05},"last":0.378},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84799999999999998,"5min":1.046,"15min":1.046},"last":0.46600000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.625},"min":{"1min":0.60799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.49299999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65000000000000002,"15min":0.61799999999999999},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85799999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.34300000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.60899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.022,"15min":1.022},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57699999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78700000000000003,"15min":0.77000000000000002},"min":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.073,"15min":1.073},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.92500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:08:39.657 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-11-17T00:08:39.657 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-17T00:08:39.740 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:39 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/775693262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:39.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:39 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/775693262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:39.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:39 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/775693262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:39.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 -- 172.21.15.6:0/3701241021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84073550 msgr2=0x7f1b840739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 --2- 172.21.15.6:0/3701241021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84073550 0x7f1b840739c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009990 tx=0x7f1b78009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 -- 172.21.15.6:0/3701241021 shutdown_connections 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 --2- 172.21.15.6:0/3701241021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84108190 0x7f1b8410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 --2- 172.21.15.6:0/3701241021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84074b30 0x7f1b84072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 --2- 172.21.15.6:0/3701241021 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84073550 0x7f1b840739c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 -- 172.21.15.6:0/3701241021 >> 172.21.15.6:0/3701241021 conn(0x7f1b840fb8b0 msgr2=0x7f1b840fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 -- 172.21.15.6:0/3701241021 shutdown_connections 2023-11-17T00:08:39.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.886+0000 7f1b89bff700 1 -- 172.21.15.6:0/3701241021 wait complete. 2023-11-17T00:08:39.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.888+0000 7f1b89bff700 1 Processor -- start 2023-11-17T00:08:39.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.888+0000 7f1b89bff700 1 -- start start 2023-11-17T00:08:39.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84073550 0x7f1b841982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 0x7f1b841987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8419fe30 con 0x7f1b84108190 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8419ff70 con 0x7f1b84074b30 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b89bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841a0100 con 0x7f1b84073550 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 0x7f1b841987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60458/0 (socket says 172.21.15.6:60458) 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 0x7f1b841987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:32910/0 (socket says 172.21.15.6:32910) 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 -- 172.21.15.6:0/224398284 learned_addr learned my addr 172.21.15.6:0/224398284 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b837fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84073550 0x7f1b841982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 -- 172.21.15.6:0/224398284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84073550 msgr2=0x7f1b841982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84073550 0x7f1b841982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 -- 172.21.15.6:0/224398284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 msgr2=0x7f1b841987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 0x7f1b841987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.889+0000 7f1b83fff700 1 -- 172.21.15.6:0/224398284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b780095f0 con 0x7f1b84108190 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b83fff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00ef30 tx=0x7f1b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c018070 con 0x7f1b84108190 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b7c00f040 con 0x7f1b84108190 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8410deb0 con 0x7f1b84108190 2023-11-17T00:08:39.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c00cd00 con 0x7f1b84108190 2023-11-17T00:08:39.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.890+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8410dd20 con 0x7f1b84108190 2023-11-17T00:08:39.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.891+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84106840 con 0x7f1b84108190 2023-11-17T00:08:39.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.893+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1b7c003a00 con 0x7f1b84108190 2023-11-17T00:08:39.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.895+0000 7f1b80ff9700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 0x7f1b6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:39.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.895+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1b7c0cc800 con 0x7f1b84108190 2023-11-17T00:08:39.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.895+0000 7f1b837fe700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 0x7f1b6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:39.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.895+0000 7f1b837fe700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 0x7f1b6c0a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b84074f00 tx=0x7f1b74009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:39.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:39.898+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b7c095c10 con 0x7f1b84108190 2023-11-17T00:08:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.104+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b84061c90 con 0x7f1b6c0a1150 2023-11-17T00:08:40.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.117+0000 7f1b80ff9700 1 -- 172.21.15.6:0/224398284 <== mgr.14148 v2:172.21.15.6:6800/879660796 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427299 (secure 0 0 0) 0x7f1b84061c90 con 0x7f1b6c0a1150 2023-11-17T00:08:40.123 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:40.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 msgr2=0x7f1b6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 0x7f1b6c0a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b84074f00 tx=0x7f1b74009260 comp rx=0 tx=0).stop 2023-11-17T00:08:40.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 msgr2=0x7f1b8419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00ef30 tx=0x7f1b7c00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 shutdown_connections 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1b6c0a1150 0x7f1b6c0a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b84108190 0x7f1b8419f860 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b84073550 0x7f1b841982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 --2- 172.21.15.6:0/224398284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b84074b30 0x7f1b841987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 >> 172.21.15.6:0/224398284 conn(0x7f1b840fb8b0 msgr2=0x7f1b840fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 shutdown_connections 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.122+0000 7f1b89bff700 1 -- 172.21.15.6:0/224398284 wait complete. 2023-11-17T00:08:40.127 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-17T00:08:40.146 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-11-17T00:08:39.778640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658778,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":396,"num_read_kb":318,"num_write":265,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82216,"kb_used_data":12168,"kb_used_omap":0,"kb_used_meta":69952,"kb_avail":1124859608,"statfs":{"total":1151940427776,"available":1151856238592,"internally_reserved":0,"allocated":12460032,"data_stored":8120814,"data_compressed":389958,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":71630848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003329"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847633+0000","last_change":"2023-11-17T00:06:12.636037+0000","last_active":"2023-11-17T00:08:03.847633+0000","last_peered":"2023-11-17T00:08:03.847633+0000","last_clean":"2023-11-17T00:08:03.847633+0000","last_became_active":"2023-11-17T00:06:12.635643+0000","last_became_peered":"2023-11-17T00:06:12.635643+0000","last_unstale":"2023-11-17T00:08:03.847633+0000","last_undegraded":"2023-11-17T00:08:03.847633+0000","last_fullsized":"2023-11-17T00:08:03.847633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:21:56.490827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850194+0000","last_change":"2023-11-17T00:06:12.638615+0000","last_active":"2023-11-17T00:08:03.850194+0000","last_peered":"2023-11-17T00:08:03.850194+0000","last_clean":"2023-11-17T00:08:03.850194+0000","last_became_active":"2023-11-17T00:06:12.632370+0000","last_became_peered":"2023-11-17T00:06:12.632370+0000","last_unstale":"2023-11-17T00:08:03.850194+0000","last_undegraded":"2023-11-17T00:08:03.850194+0000","last_fullsized":"2023-11-17T00:08:03.850194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:10:40.278758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849881+0000","last_change":"2023-11-17T00:06:12.630832+0000","last_active":"2023-11-17T00:08:03.849881+0000","last_peered":"2023-11-17T00:08:03.849881+0000","last_clean":"2023-11-17T00:08:03.849881+0000","last_became_active":"2023-11-17T00:06:12.627820+0000","last_became_peered":"2023-11-17T00:06:12.627820+0000","last_unstale":"2023-11-17T00:08:03.849881+0000","last_undegraded":"2023-11-17T00:08:03.849881+0000","last_fullsized":"2023-11-17T00:08:03.849881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:35:49.793903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.848001+0000","last_change":"2023-11-17T00:06:12.619377+0000","last_active":"2023-11-17T00:08:03.848001+0000","last_peered":"2023-11-17T00:08:03.848001+0000","last_clean":"2023-11-17T00:08:03.848001+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.848001+0000","last_undegraded":"2023-11-17T00:08:03.848001+0000","last_fullsized":"2023-11-17T00:08:03.848001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:15:46.371750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844787+0000","last_change":"2023-11-17T00:06:12.633487+0000","last_active":"2023-11-17T00:08:03.844787+0000","last_peered":"2023-11-17T00:08:03.844787+0000","last_clean":"2023-11-17T00:08:03.844787+0000","last_became_active":"2023-11-17T00:06:12.633145+0000","last_became_peered":"2023-11-17T00:06:12.633145+0000","last_unstale":"2023-11-17T00:08:03.844787+0000","last_undegraded":"2023-11-17T00:08:03.844787+0000","last_fullsized":"2023-11-17T00:08:03.844787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:27:22.122959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983403+0000","last_change":"2023-11-17T00:06:12.619487+0000","last_active":"2023-11-17T00:08:03.983403+0000","last_peered":"2023-11-17T00:08:03.983403+0000","last_clean":"2023-11-17T00:08:03.983403+0000","last_became_active":"2023-11-17T00:06:12.619002+0000","last_became_peered":"2023-11-17T00:06:12.619002+0000","last_unstale":"2023-11-17T00:08:03.983403+0000","last_undegraded":"2023-11-17T00:08:03.983403+0000","last_fullsized":"2023-11-17T00:08:03.983403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:57.117387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107998+0000","last_change":"2023-11-17T00:06:12.620386+0000","last_active":"2023-11-17T00:08:04.107998+0000","last_peered":"2023-11-17T00:08:04.107998+0000","last_clean":"2023-11-17T00:08:04.107998+0000","last_became_active":"2023-11-17T00:06:12.620025+0000","last_became_peered":"2023-11-17T00:06:12.620025+0000","last_unstale":"2023-11-17T00:08:04.107998+0000","last_undegraded":"2023-11-17T00:08:04.107998+0000","last_fullsized":"2023-11-17T00:08:04.107998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:36:57.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213578+0000","last_change":"2023-11-17T00:06:12.629776+0000","last_active":"2023-11-17T00:08:04.213578+0000","last_peered":"2023-11-17T00:08:04.213578+0000","last_clean":"2023-11-17T00:08:04.213578+0000","last_became_active":"2023-11-17T00:06:12.628279+0000","last_became_peered":"2023-11-17T00:06:12.628279+0000","last_unstale":"2023-11-17T00:08:04.213578+0000","last_undegraded":"2023-11-17T00:08:04.213578+0000","last_fullsized":"2023-11-17T00:08:04.213578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:50:48.108930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264604+0000","last_change":"2023-11-17T00:06:12.634705+0000","last_active":"2023-11-17T00:08:04.264604+0000","last_peered":"2023-11-17T00:08:04.264604+0000","last_clean":"2023-11-17T00:08:04.264604+0000","last_became_active":"2023-11-17T00:06:12.634344+0000","last_became_peered":"2023-11-17T00:06:12.634344+0000","last_unstale":"2023-11-17T00:08:04.264604+0000","last_undegraded":"2023-11-17T00:08:04.264604+0000","last_fullsized":"2023-11-17T00:08:04.264604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:08:39.015361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.984014+0000","last_change":"2023-11-17T00:06:12.631566+0000","last_active":"2023-11-17T00:08:03.984014+0000","last_peered":"2023-11-17T00:08:03.984014+0000","last_clean":"2023-11-17T00:08:03.984014+0000","last_became_active":"2023-11-17T00:06:12.631183+0000","last_became_peered":"2023-11-17T00:06:12.631183+0000","last_unstale":"2023-11-17T00:08:03.984014+0000","last_undegraded":"2023-11-17T00:08:03.984014+0000","last_fullsized":"2023-11-17T00:08:03.984014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:17:51.571212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108637+0000","last_change":"2023-11-17T00:06:12.625078+0000","last_active":"2023-11-17T00:08:04.108637+0000","last_peered":"2023-11-17T00:08:04.108637+0000","last_clean":"2023-11-17T00:08:04.108637+0000","last_became_active":"2023-11-17T00:06:12.624315+0000","last_became_peered":"2023-11-17T00:06:12.624315+0000","last_unstale":"2023-11-17T00:08:04.108637+0000","last_undegraded":"2023-11-17T00:08:04.108637+0000","last_fullsized":"2023-11-17T00:08:04.108637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:07:03.626661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847909+0000","last_change":"2023-11-17T00:06:12.619479+0000","last_active":"2023-11-17T00:08:03.847909+0000","last_peered":"2023-11-17T00:08:03.847909+0000","last_clean":"2023-11-17T00:08:03.847909+0000","last_became_active":"2023-11-17T00:06:12.619008+0000","last_became_peered":"2023-11-17T00:06:12.619008+0000","last_unstale":"2023-11-17T00:08:03.847909+0000","last_undegraded":"2023-11-17T00:08:03.847909+0000","last_fullsized":"2023-11-17T00:08:03.847909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:02.516764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846592+0000","last_change":"2023-11-17T00:06:12.636599+0000","last_active":"2023-11-17T00:08:03.846592+0000","last_peered":"2023-11-17T00:08:03.846592+0000","last_clean":"2023-11-17T00:08:03.846592+0000","last_became_active":"2023-11-17T00:06:12.635849+0000","last_became_peered":"2023-11-17T00:06:12.635849+0000","last_unstale":"2023-11-17T00:08:03.846592+0000","last_undegraded":"2023-11-17T00:08:03.846592+0000","last_fullsized":"2023-11-17T00:08:03.846592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:03.569332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849826+0000","last_change":"2023-11-17T00:06:12.620557+0000","last_active":"2023-11-17T00:08:03.849826+0000","last_peered":"2023-11-17T00:08:03.849826+0000","last_clean":"2023-11-17T00:08:03.849826+0000","last_became_active":"2023-11-17T00:06:12.618174+0000","last_became_peered":"2023-11-17T00:06:12.618174+0000","last_unstale":"2023-11-17T00:08:03.849826+0000","last_undegraded":"2023-11-17T00:08:03.849826+0000","last_fullsized":"2023-11-17T00:08:03.849826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:25:22.625141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849335+0000","last_change":"2023-11-17T00:06:12.616865+0000","last_active":"2023-11-17T00:08:03.849335+0000","last_peered":"2023-11-17T00:08:03.849335+0000","last_clean":"2023-11-17T00:08:03.849335+0000","last_became_active":"2023-11-17T00:06:12.616443+0000","last_became_peered":"2023-11-17T00:06:12.616443+0000","last_unstale":"2023-11-17T00:08:03.849335+0000","last_undegraded":"2023-11-17T00:08:03.849335+0000","last_fullsized":"2023-11-17T00:08:03.849335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:11:30.202518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263937+0000","last_change":"2023-11-17T00:06:12.622115+0000","last_active":"2023-11-17T00:08:04.263937+0000","last_peered":"2023-11-17T00:08:04.263937+0000","last_clean":"2023-11-17T00:08:04.263937+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:04.263937+0000","last_undegraded":"2023-11-17T00:08:04.263937+0000","last_fullsized":"2023-11-17T00:08:04.263937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:35:46.189504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847628+0000","last_change":"2023-11-17T00:06:12.641054+0000","last_active":"2023-11-17T00:08:03.847628+0000","last_peered":"2023-11-17T00:08:03.847628+0000","last_clean":"2023-11-17T00:08:03.847628+0000","last_became_active":"2023-11-17T00:06:12.640660+0000","last_became_peered":"2023-11-17T00:06:12.640660+0000","last_unstale":"2023-11-17T00:08:03.847628+0000","last_undegraded":"2023-11-17T00:08:03.847628+0000","last_fullsized":"2023-11-17T00:08:03.847628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:41:04.189937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852535+0000","last_change":"2023-11-17T00:06:12.627183+0000","last_active":"2023-11-17T00:08:03.852535+0000","last_peered":"2023-11-17T00:08:03.852535+0000","last_clean":"2023-11-17T00:08:03.852535+0000","last_became_active":"2023-11-17T00:06:12.626860+0000","last_became_peered":"2023-11-17T00:06:12.626860+0000","last_unstale":"2023-11-17T00:08:03.852535+0000","last_undegraded":"2023-11-17T00:08:03.852535+0000","last_fullsized":"2023-11-17T00:08:03.852535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:00:54.543190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855381+0000","last_change":"2023-11-17T00:06:12.622937+0000","last_active":"2023-11-17T00:08:03.855381+0000","last_peered":"2023-11-17T00:08:03.855381+0000","last_clean":"2023-11-17T00:08:03.855381+0000","last_became_active":"2023-11-17T00:06:12.622362+0000","last_became_peered":"2023-11-17T00:06:12.622362+0000","last_unstale":"2023-11-17T00:08:03.855381+0000","last_undegraded":"2023-11-17T00:08:03.855381+0000","last_fullsized":"2023-11-17T00:08:03.855381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:33:13.207563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855173+0000","last_change":"2023-11-17T00:06:17.102981+0000","last_active":"2023-11-17T00:08:03.855173+0000","last_peered":"2023-11-17T00:08:03.855173+0000","last_clean":"2023-11-17T00:08:03.855173+0000","last_became_active":"2023-11-17T00:06:17.102573+0000","last_became_peered":"2023-11-17T00:06:17.102573+0000","last_unstale":"2023-11-17T00:08:03.855173+0000","last_undegraded":"2023-11-17T00:08:03.855173+0000","last_fullsized":"2023-11-17T00:08:03.855173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:30:14.028665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852679+0000","last_change":"2023-11-17T00:06:12.101431+0000","last_active":"2023-11-17T00:08:03.852679+0000","last_peered":"2023-11-17T00:08:03.852679+0000","last_clean":"2023-11-17T00:08:03.852679+0000","last_became_active":"2023-11-17T00:06:12.099890+0000","last_became_peered":"2023-11-17T00:06:12.099890+0000","last_unstale":"2023-11-17T00:08:03.852679+0000","last_undegraded":"2023-11-17T00:08:03.852679+0000","last_fullsized":"2023-11-17T00:08:03.852679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:12:07.372586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857037+0000","last_change":"2023-11-17T00:08:03.856531+0000","last_active":"2023-11-17T00:08:03.857037+0000","last_peered":"2023-11-17T00:08:03.857037+0000","last_clean":"2023-11-17T00:08:03.857037+0000","last_became_active":"2023-11-17T00:08:03.855959+0000","last_became_peered":"2023-11-17T00:08:03.855959+0000","last_unstale":"2023-11-17T00:08:03.857037+0000","last_undegraded":"2023-11-17T00:08:03.857037+0000","last_fullsized":"2023-11-17T00:08:03.857037+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:14.852453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847214+0000","last_change":"2023-11-17T00:06:17.092146+0000","last_active":"2023-11-17T00:08:03.847214+0000","last_peered":"2023-11-17T00:08:03.847214+0000","last_clean":"2023-11-17T00:08:03.847214+0000","last_became_active":"2023-11-17T00:06:17.091721+0000","last_became_peered":"2023-11-17T00:06:17.091721+0000","last_unstale":"2023-11-17T00:08:03.847214+0000","last_undegraded":"2023-11-17T00:08:03.847214+0000","last_fullsized":"2023-11-17T00:08:03.847214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:51:35.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213299+0000","last_change":"2023-11-17T00:06:12.111358+0000","last_active":"2023-11-17T00:08:04.213299+0000","last_peered":"2023-11-17T00:08:04.213299+0000","last_clean":"2023-11-17T00:08:04.213299+0000","last_became_active":"2023-11-17T00:06:12.110956+0000","last_became_peered":"2023-11-17T00:06:12.110956+0000","last_unstale":"2023-11-17T00:08:04.213299+0000","last_undegraded":"2023-11-17T00:08:04.213299+0000","last_fullsized":"2023-11-17T00:08:04.213299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:08:56.426877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850061+0000","last_change":"2023-11-17T00:06:12.631819+0000","last_active":"2023-11-17T00:08:03.850061+0000","last_peered":"2023-11-17T00:08:03.850061+0000","last_clean":"2023-11-17T00:08:03.850061+0000","last_became_active":"2023-11-17T00:06:12.624656+0000","last_became_peered":"2023-11-17T00:06:12.624656+0000","last_unstale":"2023-11-17T00:08:03.850061+0000","last_undegraded":"2023-11-17T00:08:03.850061+0000","last_fullsized":"2023-11-17T00:08:03.850061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:19:55.657467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847356+0000","last_change":"2023-11-17T00:06:17.102748+0000","last_active":"2023-11-17T00:08:03.847356+0000","last_peered":"2023-11-17T00:08:03.847356+0000","last_clean":"2023-11-17T00:08:03.847356+0000","last_became_active":"2023-11-17T00:06:17.102326+0000","last_became_peered":"2023-11-17T00:06:17.102326+0000","last_unstale":"2023-11-17T00:08:03.847356+0000","last_undegraded":"2023-11-17T00:08:03.847356+0000","last_fullsized":"2023-11-17T00:08:03.847356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:40:42.046652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845271+0000","last_change":"2023-11-17T00:06:12.105092+0000","last_active":"2023-11-17T00:08:03.845271+0000","last_peered":"2023-11-17T00:08:03.845271+0000","last_clean":"2023-11-17T00:08:03.845271+0000","last_became_active":"2023-11-17T00:06:12.104676+0000","last_became_peered":"2023-11-17T00:06:12.104676+0000","last_unstale":"2023-11-17T00:08:03.845271+0000","last_undegraded":"2023-11-17T00:08:03.845271+0000","last_fullsized":"2023-11-17T00:08:03.845271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:46:34.862241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264181+0000","last_change":"2023-11-17T00:06:12.636581+0000","last_active":"2023-11-17T00:08:04.264181+0000","last_peered":"2023-11-17T00:08:04.264181+0000","last_clean":"2023-11-17T00:08:04.264181+0000","last_became_active":"2023-11-17T00:06:12.635956+0000","last_became_peered":"2023-11-17T00:06:12.635956+0000","last_unstale":"2023-11-17T00:08:04.264181+0000","last_undegraded":"2023-11-17T00:08:04.264181+0000","last_fullsized":"2023-11-17T00:08:04.264181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:42:55.876818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847221+0000","last_change":"2023-11-17T00:06:17.100290+0000","last_active":"2023-11-17T00:08:03.847221+0000","last_peered":"2023-11-17T00:08:03.847221+0000","last_clean":"2023-11-17T00:08:03.847221+0000","last_became_active":"2023-11-17T00:06:17.099775+0000","last_became_peered":"2023-11-17T00:06:17.099775+0000","last_unstale":"2023-11-17T00:08:03.847221+0000","last_undegraded":"2023-11-17T00:08:03.847221+0000","last_fullsized":"2023-11-17T00:08:03.847221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:19:29.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845331+0000","last_change":"2023-11-17T00:06:12.099579+0000","last_active":"2023-11-17T00:08:03.845331+0000","last_peered":"2023-11-17T00:08:03.845331+0000","last_clean":"2023-11-17T00:08:03.845331+0000","last_became_active":"2023-11-17T00:06:12.099199+0000","last_became_peered":"2023-11-17T00:06:12.099199+0000","last_unstale":"2023-11-17T00:08:03.845331+0000","last_undegraded":"2023-11-17T00:08:03.845331+0000","last_fullsized":"2023-11-17T00:08:03.845331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:12:06.129075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852492+0000","last_change":"2023-11-17T00:06:12.637133+0000","last_active":"2023-11-17T00:08:03.852492+0000","last_peered":"2023-11-17T00:08:03.852492+0000","last_clean":"2023-11-17T00:08:03.852492+0000","last_became_active":"2023-11-17T00:06:12.636640+0000","last_became_peered":"2023-11-17T00:06:12.636640+0000","last_unstale":"2023-11-17T00:08:03.852492+0000","last_undegraded":"2023-11-17T00:08:03.852492+0000","last_fullsized":"2023-11-17T00:08:03.852492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:52.353552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854846+0000","last_change":"2023-11-17T00:06:17.090197+0000","last_active":"2023-11-17T00:08:03.854846+0000","last_peered":"2023-11-17T00:08:03.854846+0000","last_clean":"2023-11-17T00:08:03.854846+0000","last_became_active":"2023-11-17T00:06:17.089717+0000","last_became_peered":"2023-11-17T00:06:17.089717+0000","last_unstale":"2023-11-17T00:08:03.854846+0000","last_undegraded":"2023-11-17T00:08:03.854846+0000","last_fullsized":"2023-11-17T00:08:03.854846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T12:02:25.633349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851996+0000","last_change":"2023-11-17T00:06:12.104447+0000","last_active":"2023-11-17T00:08:03.851996+0000","last_peered":"2023-11-17T00:08:03.851996+0000","last_clean":"2023-11-17T00:08:03.851996+0000","last_became_active":"2023-11-17T00:06:12.104096+0000","last_became_peered":"2023-11-17T00:06:12.104096+0000","last_unstale":"2023-11-17T00:08:03.851996+0000","last_undegraded":"2023-11-17T00:08:03.851996+0000","last_fullsized":"2023-11-17T00:08:03.851996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:05:47.233997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844846+0000","last_change":"2023-11-17T00:06:12.627367+0000","last_active":"2023-11-17T00:08:03.844846+0000","last_peered":"2023-11-17T00:08:03.844846+0000","last_clean":"2023-11-17T00:08:03.844846+0000","last_became_active":"2023-11-17T00:06:12.627004+0000","last_became_peered":"2023-11-17T00:06:12.627004+0000","last_unstale":"2023-11-17T00:08:03.844846+0000","last_undegraded":"2023-11-17T00:08:03.844846+0000","last_fullsized":"2023-11-17T00:08:03.844846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:15:29.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854671+0000","last_change":"2023-11-17T00:06:17.107661+0000","last_active":"2023-11-17T00:08:03.854671+0000","last_peered":"2023-11-17T00:08:03.854671+0000","last_clean":"2023-11-17T00:08:03.854671+0000","last_became_active":"2023-11-17T00:06:17.107266+0000","last_became_peered":"2023-11-17T00:06:17.107266+0000","last_unstale":"2023-11-17T00:08:03.854671+0000","last_undegraded":"2023-11-17T00:08:03.854671+0000","last_fullsized":"2023-11-17T00:08:03.854671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:29.359449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983372+0000","last_change":"2023-11-17T00:06:12.095119+0000","last_active":"2023-11-17T00:08:03.983372+0000","last_peered":"2023-11-17T00:08:03.983372+0000","last_clean":"2023-11-17T00:08:03.983372+0000","last_became_active":"2023-11-17T00:06:12.094768+0000","last_became_peered":"2023-11-17T00:06:12.094768+0000","last_unstale":"2023-11-17T00:08:03.983372+0000","last_undegraded":"2023-11-17T00:08:03.983372+0000","last_fullsized":"2023-11-17T00:08:03.983372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:58:34.049286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852202+0000","last_change":"2023-11-17T00:06:12.630330+0000","last_active":"2023-11-17T00:08:03.852202+0000","last_peered":"2023-11-17T00:08:03.852202+0000","last_clean":"2023-11-17T00:08:03.852202+0000","last_became_active":"2023-11-17T00:06:12.629826+0000","last_became_peered":"2023-11-17T00:06:12.629826+0000","last_unstale":"2023-11-17T00:08:03.852202+0000","last_undegraded":"2023-11-17T00:08:03.852202+0000","last_fullsized":"2023-11-17T00:08:03.852202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847506+0000","last_change":"2023-11-17T00:07:02.907032+0000","last_active":"2023-11-17T00:08:03.847506+0000","last_peered":"2023-11-17T00:08:03.847506+0000","last_clean":"2023-11-17T00:08:03.847506+0000","last_became_active":"2023-11-17T00:07:02.906576+0000","last_became_peered":"2023-11-17T00:07:02.906576+0000","last_unstale":"2023-11-17T00:08:03.847506+0000","last_undegraded":"2023-11-17T00:08:03.847506+0000","last_fullsized":"2023-11-17T00:08:03.847506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:05:54.675471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846187+0000","last_change":"2023-11-17T00:06:12.096889+0000","last_active":"2023-11-17T00:08:03.846187+0000","last_peered":"2023-11-17T00:08:03.846187+0000","last_clean":"2023-11-17T00:08:03.846187+0000","last_became_active":"2023-11-17T00:06:12.096502+0000","last_became_peered":"2023-11-17T00:06:12.096502+0000","last_unstale":"2023-11-17T00:08:03.846187+0000","last_undegraded":"2023-11-17T00:08:03.846187+0000","last_fullsized":"2023-11-17T00:08:03.846187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:41:03.642477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213484+0000","last_change":"2023-11-17T00:06:12.631202+0000","last_active":"2023-11-17T00:08:04.213484+0000","last_peered":"2023-11-17T00:08:04.213484+0000","last_clean":"2023-11-17T00:08:04.213484+0000","last_became_active":"2023-11-17T00:06:12.628921+0000","last_became_peered":"2023-11-17T00:06:12.628921+0000","last_unstale":"2023-11-17T00:08:04.213484+0000","last_undegraded":"2023-11-17T00:08:04.213484+0000","last_fullsized":"2023-11-17T00:08:04.213484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:08:04.328652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850000+0000","last_change":"2023-11-17T00:06:17.093266+0000","last_active":"2023-11-17T00:08:03.850000+0000","last_peered":"2023-11-17T00:08:03.850000+0000","last_clean":"2023-11-17T00:08:03.850000+0000","last_became_active":"2023-11-17T00:06:17.092437+0000","last_became_peered":"2023-11-17T00:06:17.092437+0000","last_unstale":"2023-11-17T00:08:03.850000+0000","last_undegraded":"2023-11-17T00:08:03.850000+0000","last_fullsized":"2023-11-17T00:08:03.850000+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:53.311913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851607+0000","last_change":"2023-11-17T00:06:12.100208+0000","last_active":"2023-11-17T00:08:03.851607+0000","last_peered":"2023-11-17T00:08:03.851607+0000","last_clean":"2023-11-17T00:08:03.851607+0000","last_became_active":"2023-11-17T00:06:12.099834+0000","last_became_peered":"2023-11-17T00:06:12.099834+0000","last_unstale":"2023-11-17T00:08:03.851607+0000","last_undegraded":"2023-11-17T00:08:03.851607+0000","last_fullsized":"2023-11-17T00:08:03.851607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:09.044455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851482+0000","last_change":"2023-11-17T00:06:12.625504+0000","last_active":"2023-11-17T00:08:03.851482+0000","last_peered":"2023-11-17T00:08:03.851482+0000","last_clean":"2023-11-17T00:08:03.851482+0000","last_became_active":"2023-11-17T00:06:12.625168+0000","last_became_peered":"2023-11-17T00:06:12.625168+0000","last_unstale":"2023-11-17T00:08:03.851482+0000","last_undegraded":"2023-11-17T00:08:03.851482+0000","last_fullsized":"2023-11-17T00:08:03.851482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:32:46.077981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213652+0000","last_change":"2023-11-17T00:06:17.090995+0000","last_active":"2023-11-17T00:08:04.213652+0000","last_peered":"2023-11-17T00:08:04.213652+0000","last_clean":"2023-11-17T00:08:04.213652+0000","last_became_active":"2023-11-17T00:06:17.090191+0000","last_became_peered":"2023-11-17T00:06:17.090191+0000","last_unstale":"2023-11-17T00:08:04.213652+0000","last_undegraded":"2023-11-17T00:08:04.213652+0000","last_fullsized":"2023-11-17T00:08:04.213652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:43:27.890099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.984013+0000","last_change":"2023-11-17T00:06:12.100216+0000","last_active":"2023-11-17T00:08:03.984013+0000","last_peered":"2023-11-17T00:08:03.984013+0000","last_clean":"2023-11-17T00:08:03.984013+0000","last_became_active":"2023-11-17T00:06:12.099865+0000","last_became_peered":"2023-11-17T00:06:12.099865+0000","last_unstale":"2023-11-17T00:08:03.984013+0000","last_undegraded":"2023-11-17T00:08:03.984013+0000","last_fullsized":"2023-11-17T00:08:03.984013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:28:16.131066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264508+0000","last_change":"2023-11-17T00:06:12.627853+0000","last_active":"2023-11-17T00:08:04.264508+0000","last_peered":"2023-11-17T00:08:04.264508+0000","last_clean":"2023-11-17T00:08:04.264508+0000","last_became_active":"2023-11-17T00:06:12.627532+0000","last_became_peered":"2023-11-17T00:06:12.627532+0000","last_unstale":"2023-11-17T00:08:04.264508+0000","last_undegraded":"2023-11-17T00:08:04.264508+0000","last_fullsized":"2023-11-17T00:08:04.264508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:38:51.404549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851073+0000","last_change":"2023-11-17T00:06:17.103522+0000","last_active":"2023-11-17T00:08:03.851073+0000","last_peered":"2023-11-17T00:08:03.851073+0000","last_clean":"2023-11-17T00:08:03.851073+0000","last_became_active":"2023-11-17T00:06:17.102601+0000","last_became_peered":"2023-11-17T00:06:17.102601+0000","last_unstale":"2023-11-17T00:08:03.851073+0000","last_undegraded":"2023-11-17T00:08:03.851073+0000","last_fullsized":"2023-11-17T00:08:03.851073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:07:36.078352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108453+0000","last_change":"2023-11-17T00:06:12.106697+0000","last_active":"2023-11-17T00:08:04.108453+0000","last_peered":"2023-11-17T00:08:04.108453+0000","last_clean":"2023-11-17T00:08:04.108453+0000","last_became_active":"2023-11-17T00:06:12.106173+0000","last_became_peered":"2023-11-17T00:06:12.106173+0000","last_unstale":"2023-11-17T00:08:04.108453+0000","last_undegraded":"2023-11-17T00:08:04.108453+0000","last_fullsized":"2023-11-17T00:08:04.108453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:11:40.146505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.853641+0000","last_change":"2023-11-17T00:08:03.853641+0000","last_active":"2023-11-17T00:08:03.853641+0000","last_peered":"2023-11-17T00:08:03.853641+0000","last_clean":"2023-11-17T00:08:03.853641+0000","last_became_active":"2023-11-17T00:08:03.852982+0000","last_became_peered":"2023-11-17T00:08:03.852982+0000","last_unstale":"2023-11-17T00:08:03.853641+0000","last_undegraded":"2023-11-17T00:08:03.853641+0000","last_fullsized":"2023-11-17T00:08:03.853641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:28.723617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847803+0000","last_change":"2023-11-17T00:06:17.102954+0000","last_active":"2023-11-17T00:08:03.847803+0000","last_peered":"2023-11-17T00:08:03.847803+0000","last_clean":"2023-11-17T00:08:03.847803+0000","last_became_active":"2023-11-17T00:06:17.102393+0000","last_became_peered":"2023-11-17T00:06:17.102393+0000","last_unstale":"2023-11-17T00:08:03.847803+0000","last_undegraded":"2023-11-17T00:08:03.847803+0000","last_fullsized":"2023-11-17T00:08:03.847803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:15:21.684199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852483+0000","last_change":"2023-11-17T00:06:12.112002+0000","last_active":"2023-11-17T00:08:03.852483+0000","last_peered":"2023-11-17T00:08:03.852483+0000","last_clean":"2023-11-17T00:08:03.852483+0000","last_became_active":"2023-11-17T00:06:12.111670+0000","last_became_peered":"2023-11-17T00:06:12.111670+0000","last_unstale":"2023-11-17T00:08:03.852483+0000","last_undegraded":"2023-11-17T00:08:03.852483+0000","last_fullsized":"2023-11-17T00:08:03.852483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:42.759384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.264242+0000","last_change":"2023-11-17T00:06:12.622828+0000","last_active":"2023-11-17T00:08:04.264242+0000","last_peered":"2023-11-17T00:08:04.264242+0000","last_clean":"2023-11-17T00:08:04.264242+0000","last_became_active":"2023-11-17T00:06:12.622514+0000","last_became_peered":"2023-11-17T00:06:12.622514+0000","last_unstale":"2023-11-17T00:08:04.264242+0000","last_undegraded":"2023-11-17T00:08:04.264242+0000","last_fullsized":"2023-11-17T00:08:04.264242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:30:52.800786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851596+0000","last_change":"2023-11-17T00:06:17.109247+0000","last_active":"2023-11-17T00:08:03.851596+0000","last_peered":"2023-11-17T00:08:03.851596+0000","last_clean":"2023-11-17T00:08:03.851596+0000","last_became_active":"2023-11-17T00:06:17.107909+0000","last_became_peered":"2023-11-17T00:06:17.107909+0000","last_unstale":"2023-11-17T00:08:03.851596+0000","last_undegraded":"2023-11-17T00:08:03.851596+0000","last_fullsized":"2023-11-17T00:08:03.851596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:37:04.648542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":320,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:38.034836+0000","last_change":"2023-11-17T00:06:12.106568+0000","last_active":"2023-11-17T00:08:38.034836+0000","last_peered":"2023-11-17T00:08:38.034836+0000","last_clean":"2023-11-17T00:08:38.034836+0000","last_became_active":"2023-11-17T00:06:12.105732+0000","last_became_peered":"2023-11-17T00:06:12.105732+0000","last_unstale":"2023-11-17T00:08:38.034836+0000","last_undegraded":"2023-11-17T00:08:38.034836+0000","last_fullsized":"2023-11-17T00:08:38.034836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:26:09.862054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67366,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":254,"num_read_kb":193,"num_write":35,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845080+0000","last_change":"2023-11-17T00:06:12.627734+0000","last_active":"2023-11-17T00:08:03.845080+0000","last_peered":"2023-11-17T00:08:03.845080+0000","last_clean":"2023-11-17T00:08:03.845080+0000","last_became_active":"2023-11-17T00:06:12.627157+0000","last_became_peered":"2023-11-17T00:06:12.627157+0000","last_unstale":"2023-11-17T00:08:03.845080+0000","last_undegraded":"2023-11-17T00:08:03.845080+0000","last_fullsized":"2023-11-17T00:08:03.845080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:53:06.594615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852873+0000","last_change":"2023-11-17T00:06:17.097470+0000","last_active":"2023-11-17T00:08:03.852873+0000","last_peered":"2023-11-17T00:08:03.852873+0000","last_clean":"2023-11-17T00:08:03.852873+0000","last_became_active":"2023-11-17T00:06:17.097071+0000","last_became_peered":"2023-11-17T00:06:17.097071+0000","last_unstale":"2023-11-17T00:08:03.852873+0000","last_undegraded":"2023-11-17T00:08:03.852873+0000","last_fullsized":"2023-11-17T00:08:03.852873+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:19:00.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:07:03.428692+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:07:02.832461+0000","last_became_peered":"2023-11-17T00:07:02.832461+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:54:18.428663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846331+0000","last_change":"2023-11-17T00:06:12.625274+0000","last_active":"2023-11-17T00:08:03.846331+0000","last_peered":"2023-11-17T00:08:03.846331+0000","last_clean":"2023-11-17T00:08:03.846331+0000","last_became_active":"2023-11-17T00:06:12.624949+0000","last_became_peered":"2023-11-17T00:06:12.624949+0000","last_unstale":"2023-11-17T00:08:03.846331+0000","last_undegraded":"2023-11-17T00:08:03.846331+0000","last_fullsized":"2023-11-17T00:08:03.846331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:33:28.090943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847175+0000","last_change":"2023-11-17T00:07:02.831797+0000","last_active":"2023-11-17T00:08:03.847175+0000","last_peered":"2023-11-17T00:08:03.847175+0000","last_clean":"2023-11-17T00:08:03.847175+0000","last_became_active":"2023-11-17T00:07:02.831339+0000","last_became_peered":"2023-11-17T00:07:02.831339+0000","last_unstale":"2023-11-17T00:08:03.847175+0000","last_undegraded":"2023-11-17T00:08:03.847175+0000","last_fullsized":"2023-11-17T00:08:03.847175+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:08:45.664170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852233+0000","last_change":"2023-11-17T00:06:12.104389+0000","last_active":"2023-11-17T00:08:03.852233+0000","last_peered":"2023-11-17T00:08:03.852233+0000","last_clean":"2023-11-17T00:08:03.852233+0000","last_became_active":"2023-11-17T00:06:12.104041+0000","last_became_peered":"2023-11-17T00:06:12.104041+0000","last_unstale":"2023-11-17T00:08:03.852233+0000","last_undegraded":"2023-11-17T00:08:03.852233+0000","last_fullsized":"2023-11-17T00:08:03.852233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:55:14.023586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108152+0000","last_change":"2023-11-17T00:07:02.905543+0000","last_active":"2023-11-17T00:08:04.108152+0000","last_peered":"2023-11-17T00:08:04.108152+0000","last_clean":"2023-11-17T00:08:04.108152+0000","last_became_active":"2023-11-17T00:07:02.905087+0000","last_became_peered":"2023-11-17T00:07:02.905087+0000","last_unstale":"2023-11-17T00:08:04.108152+0000","last_undegraded":"2023-11-17T00:08:04.108152+0000","last_fullsized":"2023-11-17T00:08:04.108152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:54:13.888941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852088+0000","last_change":"2023-11-17T00:06:17.101035+0000","last_active":"2023-11-17T00:08:03.852088+0000","last_peered":"2023-11-17T00:08:03.852088+0000","last_clean":"2023-11-17T00:08:03.852088+0000","last_became_active":"2023-11-17T00:06:17.097800+0000","last_became_peered":"2023-11-17T00:06:17.097800+0000","last_unstale":"2023-11-17T00:08:03.852088+0000","last_undegraded":"2023-11-17T00:08:03.852088+0000","last_fullsized":"2023-11-17T00:08:03.852088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:01:46.884846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847537+0000","last_change":"2023-11-17T00:06:12.103248+0000","last_active":"2023-11-17T00:08:03.847537+0000","last_peered":"2023-11-17T00:08:03.847537+0000","last_clean":"2023-11-17T00:08:03.847537+0000","last_became_active":"2023-11-17T00:06:12.102634+0000","last_became_peered":"2023-11-17T00:06:12.102634+0000","last_unstale":"2023-11-17T00:08:03.847537+0000","last_undegraded":"2023-11-17T00:08:03.847537+0000","last_fullsized":"2023-11-17T00:08:03.847537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:15:45.367954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108387+0000","last_change":"2023-11-17T00:06:12.634918+0000","last_active":"2023-11-17T00:08:04.108387+0000","last_peered":"2023-11-17T00:08:04.108387+0000","last_clean":"2023-11-17T00:08:04.108387+0000","last_became_active":"2023-11-17T00:06:12.634355+0000","last_became_peered":"2023-11-17T00:06:12.634355+0000","last_unstale":"2023-11-17T00:08:04.108387+0000","last_undegraded":"2023-11-17T00:08:04.108387+0000","last_fullsized":"2023-11-17T00:08:04.108387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:28.816560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846840+0000","last_change":"2023-11-17T00:06:17.107846+0000","last_active":"2023-11-17T00:08:03.846840+0000","last_peered":"2023-11-17T00:08:03.846840+0000","last_clean":"2023-11-17T00:08:03.846840+0000","last_became_active":"2023-11-17T00:06:17.107353+0000","last_became_peered":"2023-11-17T00:06:17.107353+0000","last_unstale":"2023-11-17T00:08:03.846840+0000","last_undegraded":"2023-11-17T00:08:03.846840+0000","last_fullsized":"2023-11-17T00:08:03.846840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:32.403943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844650+0000","last_change":"2023-11-17T00:06:12.108207+0000","last_active":"2023-11-17T00:08:03.844650+0000","last_peered":"2023-11-17T00:08:03.844650+0000","last_clean":"2023-11-17T00:08:03.844650+0000","last_became_active":"2023-11-17T00:06:12.107785+0000","last_became_peered":"2023-11-17T00:06:12.107785+0000","last_unstale":"2023-11-17T00:08:03.844650+0000","last_undegraded":"2023-11-17T00:08:03.844650+0000","last_fullsized":"2023-11-17T00:08:03.844650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:38:35.557739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851979+0000","last_change":"2023-11-17T00:06:12.633327+0000","last_active":"2023-11-17T00:08:03.851979+0000","last_peered":"2023-11-17T00:08:03.851979+0000","last_clean":"2023-11-17T00:08:03.851979+0000","last_became_active":"2023-11-17T00:06:12.632959+0000","last_became_peered":"2023-11-17T00:06:12.632959+0000","last_unstale":"2023-11-17T00:08:03.851979+0000","last_undegraded":"2023-11-17T00:08:03.851979+0000","last_fullsized":"2023-11-17T00:08:03.851979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:18:47.226898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851714+0000","last_change":"2023-11-17T00:06:17.088734+0000","last_active":"2023-11-17T00:08:03.851714+0000","last_peered":"2023-11-17T00:08:03.851714+0000","last_clean":"2023-11-17T00:08:03.851714+0000","last_became_active":"2023-11-17T00:06:17.088235+0000","last_became_peered":"2023-11-17T00:06:17.088235+0000","last_unstale":"2023-11-17T00:08:03.851714+0000","last_undegraded":"2023-11-17T00:08:03.851714+0000","last_fullsized":"2023-11-17T00:08:03.851714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:40:13.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845280+0000","last_change":"2023-11-17T00:06:12.100654+0000","last_active":"2023-11-17T00:08:03.845280+0000","last_peered":"2023-11-17T00:08:03.845280+0000","last_clean":"2023-11-17T00:08:03.845280+0000","last_became_active":"2023-11-17T00:06:12.100263+0000","last_became_peered":"2023-11-17T00:06:12.100263+0000","last_unstale":"2023-11-17T00:08:03.845280+0000","last_undegraded":"2023-11-17T00:08:03.845280+0000","last_fullsized":"2023-11-17T00:08:03.845280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T10:10:28.500794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847488+0000","last_change":"2023-11-17T00:06:12.635501+0000","last_active":"2023-11-17T00:08:03.847488+0000","last_peered":"2023-11-17T00:08:03.847488+0000","last_clean":"2023-11-17T00:08:03.847488+0000","last_became_active":"2023-11-17T00:06:12.635111+0000","last_became_peered":"2023-11-17T00:06:12.635111+0000","last_unstale":"2023-11-17T00:08:03.847488+0000","last_undegraded":"2023-11-17T00:08:03.847488+0000","last_fullsized":"2023-11-17T00:08:03.847488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:01:59.951206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844852+0000","last_change":"2023-11-17T00:06:17.115313+0000","last_active":"2023-11-17T00:08:03.844852+0000","last_peered":"2023-11-17T00:08:03.844852+0000","last_clean":"2023-11-17T00:08:03.844852+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.844852+0000","last_undegraded":"2023-11-17T00:08:03.844852+0000","last_fullsized":"2023-11-17T00:08:03.844852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:20.769706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852078+0000","last_change":"2023-11-17T00:06:12.106110+0000","last_active":"2023-11-17T00:08:03.852078+0000","last_peered":"2023-11-17T00:08:03.852078+0000","last_clean":"2023-11-17T00:08:03.852078+0000","last_became_active":"2023-11-17T00:06:12.105738+0000","last_became_peered":"2023-11-17T00:06:12.105738+0000","last_unstale":"2023-11-17T00:08:03.852078+0000","last_undegraded":"2023-11-17T00:08:03.852078+0000","last_fullsized":"2023-11-17T00:08:03.852078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:40:36.899273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983811+0000","last_change":"2023-11-17T00:06:12.636213+0000","last_active":"2023-11-17T00:08:03.983811+0000","last_peered":"2023-11-17T00:08:03.983811+0000","last_clean":"2023-11-17T00:08:03.983811+0000","last_became_active":"2023-11-17T00:06:12.635864+0000","last_became_peered":"2023-11-17T00:06:12.635864+0000","last_unstale":"2023-11-17T00:08:03.983811+0000","last_undegraded":"2023-11-17T00:08:03.983811+0000","last_fullsized":"2023-11-17T00:08:03.983811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:53:36.530566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846570+0000","last_change":"2023-11-17T00:06:17.099908+0000","last_active":"2023-11-17T00:08:03.846570+0000","last_peered":"2023-11-17T00:08:03.846570+0000","last_clean":"2023-11-17T00:08:03.846570+0000","last_became_active":"2023-11-17T00:06:17.098038+0000","last_became_peered":"2023-11-17T00:06:17.098038+0000","last_unstale":"2023-11-17T00:08:03.846570+0000","last_undegraded":"2023-11-17T00:08:03.846570+0000","last_fullsized":"2023-11-17T00:08:03.846570+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:06:01.945136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108107+0000","last_change":"2023-11-17T00:06:12.105356+0000","last_active":"2023-11-17T00:08:04.108107+0000","last_peered":"2023-11-17T00:08:04.108107+0000","last_clean":"2023-11-17T00:08:04.108107+0000","last_became_active":"2023-11-17T00:06:12.104973+0000","last_became_peered":"2023-11-17T00:06:12.104973+0000","last_unstale":"2023-11-17T00:08:04.108107+0000","last_undegraded":"2023-11-17T00:08:04.108107+0000","last_fullsized":"2023-11-17T00:08:04.108107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:07:33.000705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.983665+0000","last_change":"2023-11-17T00:06:12.623118+0000","last_active":"2023-11-17T00:08:03.983665+0000","last_peered":"2023-11-17T00:08:03.983665+0000","last_clean":"2023-11-17T00:08:03.983665+0000","last_became_active":"2023-11-17T00:06:12.622715+0000","last_became_peered":"2023-11-17T00:06:12.622715+0000","last_unstale":"2023-11-17T00:08:03.983665+0000","last_undegraded":"2023-11-17T00:08:03.983665+0000","last_fullsized":"2023-11-17T00:08:03.983665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:04:10.497970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846842+0000","last_change":"2023-11-17T00:06:17.115713+0000","last_active":"2023-11-17T00:08:03.846842+0000","last_peered":"2023-11-17T00:08:03.846842+0000","last_clean":"2023-11-17T00:08:03.846842+0000","last_became_active":"2023-11-17T00:06:17.114992+0000","last_became_peered":"2023-11-17T00:06:17.114992+0000","last_unstale":"2023-11-17T00:08:03.846842+0000","last_undegraded":"2023-11-17T00:08:03.846842+0000","last_fullsized":"2023-11-17T00:08:03.846842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:56:35.795593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851803+0000","last_change":"2023-11-17T00:06:12.632468+0000","last_active":"2023-11-17T00:08:03.851803+0000","last_peered":"2023-11-17T00:08:03.851803+0000","last_clean":"2023-11-17T00:08:03.851803+0000","last_became_active":"2023-11-17T00:06:12.632160+0000","last_became_peered":"2023-11-17T00:06:12.632160+0000","last_unstale":"2023-11-17T00:08:03.851803+0000","last_undegraded":"2023-11-17T00:08:03.851803+0000","last_fullsized":"2023-11-17T00:08:03.851803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:16:58.183436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.112511+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.112058+0000","last_became_peered":"2023-11-17T00:06:12.112058+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:12:41.066776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108595+0000","last_change":"2023-11-17T00:06:17.095117+0000","last_active":"2023-11-17T00:08:04.108595+0000","last_peered":"2023-11-17T00:08:04.108595+0000","last_clean":"2023-11-17T00:08:04.108595+0000","last_became_active":"2023-11-17T00:06:17.094732+0000","last_became_peered":"2023-11-17T00:06:17.094732+0000","last_unstale":"2023-11-17T00:08:04.108595+0000","last_undegraded":"2023-11-17T00:08:04.108595+0000","last_fullsized":"2023-11-17T00:08:04.108595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:11:30.163957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213635+0000","last_change":"2023-11-17T00:06:12.628845+0000","last_active":"2023-11-17T00:08:04.213635+0000","last_peered":"2023-11-17T00:08:04.213635+0000","last_clean":"2023-11-17T00:08:04.213635+0000","last_became_active":"2023-11-17T00:06:12.628528+0000","last_became_peered":"2023-11-17T00:06:12.628528+0000","last_unstale":"2023-11-17T00:08:04.213635+0000","last_undegraded":"2023-11-17T00:08:04.213635+0000","last_fullsized":"2023-11-17T00:08:04.213635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:23:03.217800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847624+0000","last_change":"2023-11-17T00:06:12.102297+0000","last_active":"2023-11-17T00:08:03.847624+0000","last_peered":"2023-11-17T00:08:03.847624+0000","last_clean":"2023-11-17T00:08:03.847624+0000","last_became_active":"2023-11-17T00:06:12.101899+0000","last_became_peered":"2023-11-17T00:06:12.101899+0000","last_unstale":"2023-11-17T00:08:03.847624+0000","last_undegraded":"2023-11-17T00:08:03.847624+0000","last_fullsized":"2023-11-17T00:08:03.847624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:29:08.828256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.107901+0000","last_change":"2023-11-17T00:05:02.693124+0000","last_active":"2023-11-17T00:08:04.107901+0000","last_peered":"2023-11-17T00:08:04.107901+0000","last_clean":"2023-11-17T00:08:04.107901+0000","last_became_active":"2023-11-17T00:05:02.692703+0000","last_became_peered":"2023-11-17T00:05:02.692703+0000","last_unstale":"2023-11-17T00:08:04.107901+0000","last_undegraded":"2023-11-17T00:08:04.107901+0000","last_fullsized":"2023-11-17T00:08:04.107901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:01:28.260982+0000","last_clean_scrub_stamp":"2023-11-17T00:01:28.260982+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852528+0000","last_change":"2023-11-17T00:06:17.115790+0000","last_active":"2023-11-17T00:08:03.852528+0000","last_peered":"2023-11-17T00:08:03.852528+0000","last_clean":"2023-11-17T00:08:03.852528+0000","last_became_active":"2023-11-17T00:06:17.115409+0000","last_became_peered":"2023-11-17T00:06:17.115409+0000","last_unstale":"2023-11-17T00:08:03.852528+0000","last_undegraded":"2023-11-17T00:08:03.852528+0000","last_fullsized":"2023-11-17T00:08:03.852528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:27:25.112450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846096+0000","last_change":"2023-11-17T00:06:12.629349+0000","last_active":"2023-11-17T00:08:03.846096+0000","last_peered":"2023-11-17T00:08:03.846096+0000","last_clean":"2023-11-17T00:08:03.846096+0000","last_became_active":"2023-11-17T00:06:12.629089+0000","last_became_peered":"2023-11-17T00:06:12.629089+0000","last_unstale":"2023-11-17T00:08:03.846096+0000","last_undegraded":"2023-11-17T00:08:03.846096+0000","last_fullsized":"2023-11-17T00:08:03.846096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:58:49.381194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108244+0000","last_change":"2023-11-17T00:07:02.906620+0000","last_active":"2023-11-17T00:08:04.108244+0000","last_peered":"2023-11-17T00:08:04.108244+0000","last_clean":"2023-11-17T00:08:04.108244+0000","last_became_active":"2023-11-17T00:07:02.906237+0000","last_became_peered":"2023-11-17T00:07:02.906237+0000","last_unstale":"2023-11-17T00:08:04.108244+0000","last_undegraded":"2023-11-17T00:08:04.108244+0000","last_fullsized":"2023-11-17T00:08:04.108244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:26:23.532266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847200+0000","last_change":"2023-11-17T00:06:17.104115+0000","last_active":"2023-11-17T00:08:03.847200+0000","last_peered":"2023-11-17T00:08:03.847200+0000","last_clean":"2023-11-17T00:08:03.847200+0000","last_became_active":"2023-11-17T00:06:17.103742+0000","last_became_peered":"2023-11-17T00:06:17.103742+0000","last_unstale":"2023-11-17T00:08:03.847200+0000","last_undegraded":"2023-11-17T00:08:03.847200+0000","last_fullsized":"2023-11-17T00:08:03.847200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:52.246429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.845164+0000","last_change":"2023-11-17T00:06:12.111011+0000","last_active":"2023-11-17T00:08:03.845164+0000","last_peered":"2023-11-17T00:08:03.845164+0000","last_clean":"2023-11-17T00:08:03.845164+0000","last_became_active":"2023-11-17T00:06:12.110679+0000","last_became_peered":"2023-11-17T00:06:12.110679+0000","last_unstale":"2023-11-17T00:08:03.845164+0000","last_undegraded":"2023-11-17T00:08:03.845164+0000","last_fullsized":"2023-11-17T00:08:03.845164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:57:07.339417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846667+0000","last_change":"2023-11-17T00:06:12.628129+0000","last_active":"2023-11-17T00:08:03.846667+0000","last_peered":"2023-11-17T00:08:03.846667+0000","last_clean":"2023-11-17T00:08:03.846667+0000","last_became_active":"2023-11-17T00:06:12.623967+0000","last_became_peered":"2023-11-17T00:06:12.623967+0000","last_unstale":"2023-11-17T00:08:03.846667+0000","last_undegraded":"2023-11-17T00:08:03.846667+0000","last_fullsized":"2023-11-17T00:08:03.846667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:53:45.282863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213467+0000","last_change":"2023-11-17T00:06:17.116664+0000","last_active":"2023-11-17T00:08:04.213467+0000","last_peered":"2023-11-17T00:08:04.213467+0000","last_clean":"2023-11-17T00:08:04.213467+0000","last_became_active":"2023-11-17T00:06:17.114968+0000","last_became_peered":"2023-11-17T00:06:17.114968+0000","last_unstale":"2023-11-17T00:08:04.213467+0000","last_undegraded":"2023-11-17T00:08:04.213467+0000","last_fullsized":"2023-11-17T00:08:04.213467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:43:40.186246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:21.576683+0000","last_change":"2023-11-17T00:06:12.096659+0000","last_active":"2023-11-17T00:08:21.576683+0000","last_peered":"2023-11-17T00:08:21.576683+0000","last_clean":"2023-11-17T00:08:21.576683+0000","last_became_active":"2023-11-17T00:06:12.096315+0000","last_became_peered":"2023-11-17T00:06:12.096315+0000","last_unstale":"2023-11-17T00:08:21.576683+0000","last_undegraded":"2023-11-17T00:08:21.576683+0000","last_fullsized":"2023-11-17T00:08:21.576683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:51.839878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214495+0000","last_change":"2023-11-17T00:06:12.622713+0000","last_active":"2023-11-17T00:08:04.214495+0000","last_peered":"2023-11-17T00:08:04.214495+0000","last_clean":"2023-11-17T00:08:04.214495+0000","last_became_active":"2023-11-17T00:06:12.622395+0000","last_became_peered":"2023-11-17T00:06:12.622395+0000","last_unstale":"2023-11-17T00:08:04.214495+0000","last_undegraded":"2023-11-17T00:08:04.214495+0000","last_fullsized":"2023-11-17T00:08:04.214495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:35:18.102221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855170+0000","last_change":"2023-11-17T00:06:17.085912+0000","last_active":"2023-11-17T00:08:03.855170+0000","last_peered":"2023-11-17T00:08:03.855170+0000","last_clean":"2023-11-17T00:08:03.855170+0000","last_became_active":"2023-11-17T00:06:17.085546+0000","last_became_peered":"2023-11-17T00:06:17.085546+0000","last_unstale":"2023-11-17T00:08:03.855170+0000","last_undegraded":"2023-11-17T00:08:03.855170+0000","last_fullsized":"2023-11-17T00:08:03.855170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:17:46.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847789+0000","last_change":"2023-11-17T00:06:12.100661+0000","last_active":"2023-11-17T00:08:03.847789+0000","last_peered":"2023-11-17T00:08:03.847789+0000","last_clean":"2023-11-17T00:08:03.847789+0000","last_became_active":"2023-11-17T00:06:12.100239+0000","last_became_peered":"2023-11-17T00:06:12.100239+0000","last_unstale":"2023-11-17T00:08:03.847789+0000","last_undegraded":"2023-11-17T00:08:03.847789+0000","last_fullsized":"2023-11-17T00:08:03.847789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:30:22.305069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.850179+0000","last_change":"2023-11-17T00:06:12.621638+0000","last_active":"2023-11-17T00:08:03.850179+0000","last_peered":"2023-11-17T00:08:03.850179+0000","last_clean":"2023-11-17T00:08:03.850179+0000","last_became_active":"2023-11-17T00:06:12.619782+0000","last_became_peered":"2023-11-17T00:06:12.619782+0000","last_unstale":"2023-11-17T00:08:03.850179+0000","last_undegraded":"2023-11-17T00:08:03.850179+0000","last_fullsized":"2023-11-17T00:08:03.850179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:24:04.518190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847548+0000","last_change":"2023-11-17T00:06:17.115878+0000","last_active":"2023-11-17T00:08:03.847548+0000","last_peered":"2023-11-17T00:08:03.847548+0000","last_clean":"2023-11-17T00:08:03.847548+0000","last_became_active":"2023-11-17T00:06:17.115476+0000","last_became_peered":"2023-11-17T00:06:17.115476+0000","last_unstale":"2023-11-17T00:08:03.847548+0000","last_undegraded":"2023-11-17T00:08:03.847548+0000","last_fullsized":"2023-11-17T00:08:03.847548+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:51.426856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263432+0000","last_change":"2023-11-17T00:06:12.106485+0000","last_active":"2023-11-17T00:08:04.263432+0000","last_peered":"2023-11-17T00:08:04.263432+0000","last_clean":"2023-11-17T00:08:04.263432+0000","last_became_active":"2023-11-17T00:06:12.105975+0000","last_became_peered":"2023-11-17T00:06:12.105975+0000","last_unstale":"2023-11-17T00:08:04.263432+0000","last_undegraded":"2023-11-17T00:08:04.263432+0000","last_fullsized":"2023-11-17T00:08:04.263432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:18:23.270940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844615+0000","last_change":"2023-11-17T00:06:12.626598+0000","last_active":"2023-11-17T00:08:03.844615+0000","last_peered":"2023-11-17T00:08:03.844615+0000","last_clean":"2023-11-17T00:08:03.844615+0000","last_became_active":"2023-11-17T00:06:12.626080+0000","last_became_peered":"2023-11-17T00:06:12.626080+0000","last_unstale":"2023-11-17T00:08:03.844615+0000","last_undegraded":"2023-11-17T00:08:03.844615+0000","last_fullsized":"2023-11-17T00:08:03.844615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:24:09.066593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852838+0000","last_change":"2023-11-17T00:06:17.105517+0000","last_active":"2023-11-17T00:08:03.852838+0000","last_peered":"2023-11-17T00:08:03.852838+0000","last_clean":"2023-11-17T00:08:03.852838+0000","last_became_active":"2023-11-17T00:06:17.101957+0000","last_became_peered":"2023-11-17T00:06:17.101957+0000","last_unstale":"2023-11-17T00:08:03.852838+0000","last_undegraded":"2023-11-17T00:08:03.852838+0000","last_fullsized":"2023-11-17T00:08:03.852838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:47.750149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213289+0000","last_change":"2023-11-17T00:06:12.091337+0000","last_active":"2023-11-17T00:08:04.213289+0000","last_peered":"2023-11-17T00:08:04.213289+0000","last_clean":"2023-11-17T00:08:04.213289+0000","last_became_active":"2023-11-17T00:06:12.090987+0000","last_became_peered":"2023-11-17T00:06:12.090987+0000","last_unstale":"2023-11-17T00:08:04.213289+0000","last_undegraded":"2023-11-17T00:08:04.213289+0000","last_fullsized":"2023-11-17T00:08:04.213289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:13:03.414872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846881+0000","last_change":"2023-11-17T00:06:12.636800+0000","last_active":"2023-11-17T00:08:03.846881+0000","last_peered":"2023-11-17T00:08:03.846881+0000","last_clean":"2023-11-17T00:08:03.846881+0000","last_became_active":"2023-11-17T00:06:12.636369+0000","last_became_peered":"2023-11-17T00:06:12.636369+0000","last_unstale":"2023-11-17T00:08:03.846881+0000","last_undegraded":"2023-11-17T00:08:03.846881+0000","last_fullsized":"2023-11-17T00:08:03.846881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:12:45.564310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847561+0000","last_change":"2023-11-17T00:06:17.102683+0000","last_active":"2023-11-17T00:08:03.847561+0000","last_peered":"2023-11-17T00:08:03.847561+0000","last_clean":"2023-11-17T00:08:03.847561+0000","last_became_active":"2023-11-17T00:06:17.102272+0000","last_became_peered":"2023-11-17T00:06:17.102272+0000","last_unstale":"2023-11-17T00:08:03.847561+0000","last_undegraded":"2023-11-17T00:08:03.847561+0000","last_fullsized":"2023-11-17T00:08:03.847561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:15:02.825415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846923+0000","last_change":"2023-11-17T00:07:02.831541+0000","last_active":"2023-11-17T00:08:03.846923+0000","last_peered":"2023-11-17T00:08:03.846923+0000","last_clean":"2023-11-17T00:08:03.846923+0000","last_became_active":"2023-11-17T00:07:02.831030+0000","last_became_peered":"2023-11-17T00:07:02.831030+0000","last_unstale":"2023-11-17T00:08:03.846923+0000","last_undegraded":"2023-11-17T00:08:03.846923+0000","last_fullsized":"2023-11-17T00:08:03.846923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:54:41.302381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214157+0000","last_change":"2023-11-17T00:06:12.623836+0000","last_active":"2023-11-17T00:08:04.214157+0000","last_peered":"2023-11-17T00:08:04.214157+0000","last_clean":"2023-11-17T00:08:04.214157+0000","last_became_active":"2023-11-17T00:06:12.623512+0000","last_became_peered":"2023-11-17T00:06:12.623512+0000","last_unstale":"2023-11-17T00:08:04.214157+0000","last_undegraded":"2023-11-17T00:08:04.214157+0000","last_fullsized":"2023-11-17T00:08:04.214157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:10:32.041314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847519+0000","last_change":"2023-11-17T00:06:17.115378+0000","last_active":"2023-11-17T00:08:03.847519+0000","last_peered":"2023-11-17T00:08:03.847519+0000","last_clean":"2023-11-17T00:08:03.847519+0000","last_became_active":"2023-11-17T00:06:17.114769+0000","last_became_peered":"2023-11-17T00:06:17.114769+0000","last_unstale":"2023-11-17T00:08:03.847519+0000","last_undegraded":"2023-11-17T00:08:03.847519+0000","last_fullsized":"2023-11-17T00:08:03.847519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:50:38.729014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844771+0000","last_change":"2023-11-17T00:06:12.103127+0000","last_active":"2023-11-17T00:08:03.844771+0000","last_peered":"2023-11-17T00:08:03.844771+0000","last_clean":"2023-11-17T00:08:03.844771+0000","last_became_active":"2023-11-17T00:06:12.102775+0000","last_became_peered":"2023-11-17T00:06:12.102775+0000","last_unstale":"2023-11-17T00:08:03.844771+0000","last_undegraded":"2023-11-17T00:08:03.844771+0000","last_fullsized":"2023-11-17T00:08:03.844771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:02.714237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108213+0000","last_change":"2023-11-17T00:06:12.632802+0000","last_active":"2023-11-17T00:08:04.108213+0000","last_peered":"2023-11-17T00:08:04.108213+0000","last_clean":"2023-11-17T00:08:04.108213+0000","last_became_active":"2023-11-17T00:06:12.632463+0000","last_became_peered":"2023-11-17T00:06:12.632463+0000","last_unstale":"2023-11-17T00:08:04.108213+0000","last_undegraded":"2023-11-17T00:08:04.108213+0000","last_fullsized":"2023-11-17T00:08:04.108213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:45:06.851039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263797+0000","last_change":"2023-11-17T00:06:17.104811+0000","last_active":"2023-11-17T00:08:04.263797+0000","last_peered":"2023-11-17T00:08:04.263797+0000","last_clean":"2023-11-17T00:08:04.263797+0000","last_became_active":"2023-11-17T00:06:17.104326+0000","last_became_peered":"2023-11-17T00:06:17.104326+0000","last_unstale":"2023-11-17T00:08:04.263797+0000","last_undegraded":"2023-11-17T00:08:04.263797+0000","last_fullsized":"2023-11-17T00:08:04.263797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T05:04:08.745867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854990+0000","last_change":"2023-11-17T00:06:12.111188+0000","last_active":"2023-11-17T00:08:03.854990+0000","last_peered":"2023-11-17T00:08:03.854990+0000","last_clean":"2023-11-17T00:08:03.854990+0000","last_became_active":"2023-11-17T00:06:12.110788+0000","last_became_peered":"2023-11-17T00:06:12.110788+0000","last_unstale":"2023-11-17T00:08:03.854990+0000","last_undegraded":"2023-11-17T00:08:03.854990+0000","last_fullsized":"2023-11-17T00:08:03.854990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:44:47.738117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108772+0000","last_change":"2023-11-17T00:06:12.622927+0000","last_active":"2023-11-17T00:08:04.108772+0000","last_peered":"2023-11-17T00:08:04.108772+0000","last_clean":"2023-11-17T00:08:04.108772+0000","last_became_active":"2023-11-17T00:06:12.622578+0000","last_became_peered":"2023-11-17T00:06:12.622578+0000","last_unstale":"2023-11-17T00:08:04.108772+0000","last_undegraded":"2023-11-17T00:08:04.108772+0000","last_fullsized":"2023-11-17T00:08:04.108772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T09:30:26.916829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849466+0000","last_change":"2023-11-17T00:07:02.906790+0000","last_active":"2023-11-17T00:08:03.849466+0000","last_peered":"2023-11-17T00:08:03.849466+0000","last_clean":"2023-11-17T00:08:03.849466+0000","last_became_active":"2023-11-17T00:07:02.906439+0000","last_became_peered":"2023-11-17T00:07:02.906439+0000","last_unstale":"2023-11-17T00:08:03.849466+0000","last_undegraded":"2023-11-17T00:08:03.849466+0000","last_fullsized":"2023-11-17T00:08:03.849466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:44:33.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.852675+0000","last_change":"2023-11-17T00:06:12.105655+0000","last_active":"2023-11-17T00:08:03.852675+0000","last_peered":"2023-11-17T00:08:03.852675+0000","last_clean":"2023-11-17T00:08:03.852675+0000","last_became_active":"2023-11-17T00:06:12.105314+0000","last_became_peered":"2023-11-17T00:06:12.105314+0000","last_unstale":"2023-11-17T00:08:03.852675+0000","last_undegraded":"2023-11-17T00:08:03.852675+0000","last_fullsized":"2023-11-17T00:08:03.852675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:09:45.189368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108257+0000","last_change":"2023-11-17T00:06:12.621321+0000","last_active":"2023-11-17T00:08:04.108257+0000","last_peered":"2023-11-17T00:08:04.108257+0000","last_clean":"2023-11-17T00:08:04.108257+0000","last_became_active":"2023-11-17T00:06:12.620904+0000","last_became_peered":"2023-11-17T00:06:12.620904+0000","last_unstale":"2023-11-17T00:08:04.108257+0000","last_undegraded":"2023-11-17T00:08:04.108257+0000","last_fullsized":"2023-11-17T00:08:04.108257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:20:34.614672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.844600+0000","last_change":"2023-11-17T00:07:02.906389+0000","last_active":"2023-11-17T00:08:03.844600+0000","last_peered":"2023-11-17T00:08:03.844600+0000","last_clean":"2023-11-17T00:08:03.844600+0000","last_became_active":"2023-11-17T00:07:02.905885+0000","last_became_peered":"2023-11-17T00:07:02.905885+0000","last_unstale":"2023-11-17T00:08:03.844600+0000","last_undegraded":"2023-11-17T00:08:03.844600+0000","last_fullsized":"2023-11-17T00:08:03.844600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:16.061510+0000","last_clean_scrub_stamp":"2023-11-17T00:06:16.061510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:46:29.740711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846887+0000","last_change":"2023-11-17T00:07:03.431743+0000","last_active":"2023-11-17T00:08:03.846887+0000","last_peered":"2023-11-17T00:08:03.846887+0000","last_clean":"2023-11-17T00:08:03.846887+0000","last_became_active":"2023-11-17T00:07:02.907841+0000","last_became_peered":"2023-11-17T00:07:02.907841+0000","last_unstale":"2023-11-17T00:08:03.846887+0000","last_undegraded":"2023-11-17T00:08:03.846887+0000","last_fullsized":"2023-11-17T00:08:03.846887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:11.075170+0000","last_clean_scrub_stamp":"2023-11-17T00:06:11.075170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:57:52.605327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.108798+0000","last_change":"2023-11-17T00:06:12.624367+0000","last_active":"2023-11-17T00:08:04.108798+0000","last_peered":"2023-11-17T00:08:04.108798+0000","last_clean":"2023-11-17T00:08:04.108798+0000","last_became_active":"2023-11-17T00:06:12.624009+0000","last_became_peered":"2023-11-17T00:06:12.624009+0000","last_unstale":"2023-11-17T00:08:04.108798+0000","last_undegraded":"2023-11-17T00:08:04.108798+0000","last_fullsized":"2023-11-17T00:08:04.108798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:57:19.682430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846992+0000","last_change":"2023-11-17T00:06:12.634966+0000","last_active":"2023-11-17T00:08:03.846992+0000","last_peered":"2023-11-17T00:08:03.846992+0000","last_clean":"2023-11-17T00:08:03.846992+0000","last_became_active":"2023-11-17T00:06:12.634600+0000","last_became_peered":"2023-11-17T00:06:12.634600+0000","last_unstale":"2023-11-17T00:08:03.846992+0000","last_undegraded":"2023-11-17T00:08:03.846992+0000","last_fullsized":"2023-11-17T00:08:03.846992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:52.792686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855451+0000","last_change":"2023-11-17T00:07:02.832453+0000","last_active":"2023-11-17T00:08:03.855451+0000","last_peered":"2023-11-17T00:08:03.855451+0000","last_clean":"2023-11-17T00:08:03.855451+0000","last_became_active":"2023-11-17T00:07:02.831920+0000","last_became_peered":"2023-11-17T00:07:02.831920+0000","last_unstale":"2023-11-17T00:08:03.855451+0000","last_undegraded":"2023-11-17T00:08:03.855451+0000","last_fullsized":"2023-11-17T00:08:03.855451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:17:01.346164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.854744+0000","last_change":"2023-11-17T00:06:12.618089+0000","last_active":"2023-11-17T00:08:03.854744+0000","last_peered":"2023-11-17T00:08:03.854744+0000","last_clean":"2023-11-17T00:08:03.854744+0000","last_became_active":"2023-11-17T00:06:12.617758+0000","last_became_peered":"2023-11-17T00:06:12.617758+0000","last_unstale":"2023-11-17T00:08:03.854744+0000","last_undegraded":"2023-11-17T00:08:03.854744+0000","last_fullsized":"2023-11-17T00:08:03.854744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:49:57.451212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.849775+0000","last_change":"2023-11-17T00:06:12.637543+0000","last_active":"2023-11-17T00:08:03.849775+0000","last_peered":"2023-11-17T00:08:03.849775+0000","last_clean":"2023-11-17T00:08:03.849775+0000","last_became_active":"2023-11-17T00:06:12.634609+0000","last_became_peered":"2023-11-17T00:06:12.634609+0000","last_unstale":"2023-11-17T00:08:03.849775+0000","last_undegraded":"2023-11-17T00:08:03.849775+0000","last_fullsized":"2023-11-17T00:08:03.849775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:52:19.420736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846718+0000","last_change":"2023-11-17T00:06:12.628350+0000","last_active":"2023-11-17T00:08:03.846718+0000","last_peered":"2023-11-17T00:08:03.846718+0000","last_clean":"2023-11-17T00:08:03.846718+0000","last_became_active":"2023-11-17T00:06:12.621790+0000","last_became_peered":"2023-11-17T00:06:12.621790+0000","last_unstale":"2023-11-17T00:08:03.846718+0000","last_undegraded":"2023-11-17T00:08:03.846718+0000","last_fullsized":"2023-11-17T00:08:03.846718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:22:40.198134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.857463+0000","last_change":"2023-11-17T00:08:03.856824+0000","last_active":"2023-11-17T00:08:03.857463+0000","last_peered":"2023-11-17T00:08:03.857463+0000","last_clean":"2023-11-17T00:08:03.857463+0000","last_became_active":"2023-11-17T00:08:03.856253+0000","last_became_peered":"2023-11-17T00:08:03.856253+0000","last_unstale":"2023-11-17T00:08:03.857463+0000","last_undegraded":"2023-11-17T00:08:03.857463+0000","last_fullsized":"2023-11-17T00:08:03.857463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:56:01.187155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.855315+0000","last_change":"2023-11-17T00:06:12.621246+0000","last_active":"2023-11-17T00:08:03.855315+0000","last_peered":"2023-11-17T00:08:03.855315+0000","last_clean":"2023-11-17T00:08:03.855315+0000","last_became_active":"2023-11-17T00:06:12.620937+0000","last_became_peered":"2023-11-17T00:06:12.620937+0000","last_unstale":"2023-11-17T00:08:03.855315+0000","last_undegraded":"2023-11-17T00:08:03.855315+0000","last_fullsized":"2023-11-17T00:08:03.855315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T08:23:24.316477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.214533+0000","last_change":"2023-11-17T00:06:12.611149+0000","last_active":"2023-11-17T00:08:04.214533+0000","last_peered":"2023-11-17T00:08:04.214533+0000","last_clean":"2023-11-17T00:08:04.214533+0000","last_became_active":"2023-11-17T00:06:12.609845+0000","last_became_peered":"2023-11-17T00:06:12.609845+0000","last_unstale":"2023-11-17T00:08:04.214533+0000","last_undegraded":"2023-11-17T00:08:04.214533+0000","last_fullsized":"2023-11-17T00:08:04.214533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T07:00:40.244245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.851683+0000","last_change":"2023-11-17T00:06:12.635321+0000","last_active":"2023-11-17T00:08:03.851683+0000","last_peered":"2023-11-17T00:08:03.851683+0000","last_clean":"2023-11-17T00:08:03.851683+0000","last_became_active":"2023-11-17T00:06:12.634835+0000","last_became_peered":"2023-11-17T00:06:12.634835+0000","last_unstale":"2023-11-17T00:08:03.851683+0000","last_undegraded":"2023-11-17T00:08:03.851683+0000","last_fullsized":"2023-11-17T00:08:03.851683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:02.574283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.213608+0000","last_change":"2023-11-17T00:06:12.625940+0000","last_active":"2023-11-17T00:08:04.213608+0000","last_peered":"2023-11-17T00:08:04.213608+0000","last_clean":"2023-11-17T00:08:04.213608+0000","last_became_active":"2023-11-17T00:06:12.625649+0000","last_became_peered":"2023-11-17T00:06:12.625649+0000","last_unstale":"2023-11-17T00:08:04.213608+0000","last_undegraded":"2023-11-17T00:08:04.213608+0000","last_fullsized":"2023-11-17T00:08:04.213608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:22:46.661532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:04.263993+0000","last_change":"2023-11-17T00:06:12.620778+0000","last_active":"2023-11-17T00:08:04.263993+0000","last_peered":"2023-11-17T00:08:04.263993+0000","last_clean":"2023-11-17T00:08:04.263993+0000","last_became_active":"2023-11-17T00:06:12.620388+0000","last_became_peered":"2023-11-17T00:06:12.620388+0000","last_unstale":"2023-11-17T00:08:04.263993+0000","last_undegraded":"2023-11-17T00:08:04.263993+0000","last_fullsized":"2023-11-17T00:08:04.263993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T11:32:43.430227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.846494+0000","last_change":"2023-11-17T00:06:12.621416+0000","last_active":"2023-11-17T00:08:03.846494+0000","last_peered":"2023-11-17T00:08:03.846494+0000","last_clean":"2023-11-17T00:08:03.846494+0000","last_became_active":"2023-11-17T00:06:12.620988+0000","last_became_peered":"2023-11-17T00:06:12.620988+0000","last_unstale":"2023-11-17T00:08:03.846494+0000","last_undegraded":"2023-11-17T00:08:03.846494+0000","last_fullsized":"2023-11-17T00:08:03.846494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:56.341805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-17T00:08:03.847266+0000","last_change":"2023-11-17T00:06:12.630053+0000","last_active":"2023-11-17T00:08:03.847266+0000","last_peered":"2023-11-17T00:08:03.847266+0000","last_clean":"2023-11-17T00:08:03.847266+0000","last_became_active":"2023-11-17T00:06:12.629758+0000","last_became_peered":"2023-11-17T00:06:12.629758+0000","last_unstale":"2023-11-17T00:08:03.847266+0000","last_undegraded":"2023-11-17T00:08:03.847266+0000","last_fullsized":"2023-11-17T00:08:03.847266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-17T00:06:12.077551+0000","last_clean_scrub_stamp":"2023-11-17T00:06:12.077551+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T06:00:02.220886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247522,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68164,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":260,"num_read_kb":199,"num_write":82,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":1007616,"data_stored":513556,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.39400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86199999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.042,"15min":1.042},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.53900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.75800000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.42199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78700000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.41999999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.69199999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.495},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.54100000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.505,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.432}]},{"osd":6,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.52700000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.621,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91200000000000003,"5min":1.153,"15min":1.153},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.64300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.95199999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.58699999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93000000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63},{"interface":"front","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78200000000000003,"15min":0.77300000000000002},"min":{"1min":0.66100000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1000000000000001,"5min":1.476,"15min":1.476},"last":0.624},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.77900000000000003},"min":{"1min":0.755,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98699999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67100000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.71099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94299999999999995,"5min":1.429,"15min":1.429},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.522,"15min":1.522},"last":0.58899999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":1134592,"data_stored":1167023,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45900000000000002,"15min":0.47199999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.433,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56200000000000006,"5min":0.872,"15min":0.872},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81899999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60499999999999998,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78600000000000003,"5min":0.874,"15min":0.874},"last":0.755}]},{"osd":4,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.69699999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67900000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.48999999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.879}]},{"osd":6,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.70199999999999996},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84799999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83799999999999997}]},{"osd":7,"last update":"Fri Nov 17 00:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49399999999999999}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.755,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81699999999999995,"5min":0.996,"15min":0.996},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.76300000000000001},"min":{"1min":0.58499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.5169999999999999,"5min":3.5169999999999999,"15min":3.5169999999999999},"last":0.78400000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.61599999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74199999999999999,"15min":0.73999999999999999},"min":{"1min":0.59699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.92400000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6372,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738780,"statfs":{"total":95995035648,"available":95988510720,"internally_reserved":0,"allocated":1011712,"data_stored":518200,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.622},"min":{"1min":0.35299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.47499999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.622}]},{"osd":3,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.79500000000000004,"15min":0.82999999999999996},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82999999999999996,"15min":0.84999999999999998},"min":{"1min":0.57099999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.175,"5min":0.91100000000000003,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.0700000000000003,"5min":6.0700000000000003,"15min":6.0700000000000003},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92500000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89900000000000002,"15min":0.94699999999999995},"min":{"1min":0.61599999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.629}]},{"osd":8,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.0189999999999999,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0519999999999996,"5min":6.0519999999999996,"15min":6.0519999999999996},"last":0.75}]},{"osd":10,"last update":"Fri Nov 17 00:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.91200000000000003,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98899999999999999,"5min":1.677,"15min":1.677},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93899999999999995,"15min":0.98499999999999999},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.77400000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.91500000000000004,"15min":0.93000000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1659999999999999,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1870000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.74199999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":514126,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98999999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.52100000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.83299999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.751}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.50800000000000001},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75800000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76000000000000001,"5min":1.139,"15min":1.139},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66300000000000003,"15min":0.64200000000000002},"min":{"1min":0.56100000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.308,"15min":0.308},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.62},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95599999999999996,"5min":1.075,"15min":1.075},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92700000000000005,"5min":1.008,"15min":1.008},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.014,"5min":1.022,"15min":1.022},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77000000000000002,"15min":0.74399999999999999},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72799999999999998,"15min":0.71799999999999997},"min":{"1min":0.51200000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0329999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.81499999999999995,"15min":0.79000000000000004},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78600000000000003,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0629999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76300000000000001}]},{"osd":11,"last update":"Fri Nov 17 00:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.72299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.125,"15min":1.125},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":1060864,"data_stored":1108568,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.37}]},{"osd":1,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.45400000000000001},"min":{"1min":0.317,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59399999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.433},"min":{"1min":0.375,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.627,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.70499999999999996,"5min":1.083,"15min":1.083},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.58199999999999996},"min":{"1min":0.35299999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68500000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.52700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":4.7720000000000002,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66600000000000004,"15min":0.69299999999999995},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.173,"15min":1.173},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84699999999999998,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.85299999999999998}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.68300000000000005},"min":{"1min":0.58399999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91900000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.67200000000000004},"min":{"1min":0.56499999999999995,"5min":0.441,"15min":0.441},"max":{"1min":0.92900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.752},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.751,"15min":0.745},"min":{"1min":0.63100000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86899999999999999}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82399999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.94399999999999995}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81200000000000006,"15min":0.81100000000000005},"min":{"1min":0.70099999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.92300000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":999424,"data_stored":509426,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.224,"15min":1.224},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70399999999999996,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92300000000000004,"5min":1.091,"15min":1.091},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.61299999999999999,"15min":0.65000000000000002},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75900000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.41299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.84399999999999997}]},{"osd":4,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.752,"15min":0.751},"min":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94899999999999995,"5min":1.262,"15min":1.262},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92800000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Nov 17 00:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.751},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82099999999999995,"15min":0.77000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.27,"5min":1.377,"15min":1.377},"last":0.57599999999999996}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.85399999999999998,"15min":0.85199999999999998},"min":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1240000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.89900000000000002},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.81399999999999995}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.232,"5min":1.429,"15min":1.429},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.94899999999999995,"15min":0.98499999999999999},"min":{"1min":0.68799999999999994,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.89700000000000002}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98199999999999998,"15min":1.0109999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95699999999999996,"15min":0.98499999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.0249999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.83499999999999996}]},{"osd":11,"last update":"Fri Nov 17 00:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.92800000000000005,"15min":0.93200000000000005},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.0940000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0620000000000001,"5min":1.823,"15min":1.823},"last":0.92100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7652,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737500,"statfs":{"total":95995035648,"available":95987200000,"internally_reserved":0,"allocated":1077248,"data_stored":576792,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.30499999999999999,"15min":0.27100000000000002},"max":{"1min":0.86899999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.53200000000000003},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.34200000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.501,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.84399999999999997,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.55500000000000005},"min":{"1min":0.45700000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66900000000000004,"15min":0.59099999999999997},"min":{"1min":0.50600000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88200000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624}]},{"osd":4,"last update":"Fri Nov 17 00:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.42199999999999999},"min":{"1min":0.38400000000000001,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.67900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.42099999999999999},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":1.365,"15min":1.365},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.63500000000000001,"15min":0.57299999999999995},"min":{"1min":0.57299999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88500000000000001,"5min":1.331,"15min":1.331},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Nov 17 00:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.52900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":1.395,"15min":1.395},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.502,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.501,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94999999999999996,"5min":1.155,"15min":1.155},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.67700000000000005,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.73499999999999999,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.747}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.58299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.423,"15min":1.423},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.52400000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.69999999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.0700000000000001,"5min":1.085,"15min":1.085},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97099999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Nov 17 00:08:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.72599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.77200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7452,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737700,"statfs":{"total":95995035648,"available":95987404800,"internally_reserved":0,"allocated":1003520,"data_stored":509516,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":1.385,"15min":1.385},"last":0.62},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Nov 17 00:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66500000000000004,"15min":0.60599999999999998},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.68000000000000005,"15min":0.60399999999999998},"min":{"1min":0.42199999999999999,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.373},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0940000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.747,"15min":0.61799999999999999},"min":{"1min":0.66400000000000003,"5min":0.436,"15min":0.35399999999999998},"max":{"1min":1.1459999999999999,"5min":1.675,"15min":1.675},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.53000000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Nov 17 00:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.76000000000000001,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.01}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2609999999999999,"5min":1.73,"15min":1.73},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.88600000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.90800000000000003},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1299999999999999,"5min":1.917,"15min":1.917},"last":0.93200000000000005}]},{"osd":8,"last update":"Fri Nov 17 00:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.294,"5min":1.377,"15min":1.377},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.876,"15min":0.82199999999999995},"min":{"1min":0.65800000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.048}]},{"osd":9,"last update":"Fri Nov 17 00:07:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.878,"15min":0.872},"min":{"1min":0.70399999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90400000000000003,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66700000000000004}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.149,"15min":1.232},"min":{"1min":0.71199999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.843,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92700000000000005,"15min":0.93700000000000006},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2230000000000001,"5min":1.377,"15min":1.377},"last":0.71499999999999997}]},{"osd":11,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.97099999999999997,"15min":0.95699999999999996},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.23,"5min":4.343,"15min":4.343},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.327,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.081}]}]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7652,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737500,"statfs":{"total":95995035648,"available":95987200000,"internally_reserved":0,"allocated":1077248,"data_stored":576792,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.32900000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56699999999999995,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.38,"5min":0.376,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.47999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75900000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.45100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.45300000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51900000000000002,"15min":0.47499999999999998},"min":{"1min":0.435,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.84499999999999997,"5min":1.071,"15min":1.071},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.60099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83699999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.85099999999999998},"min":{"1min":0.66400000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64300000000000002,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94399999999999995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.56599999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.751,"15min":0.72699999999999998},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96599999999999997,"5min":1.091,"15min":1.091},"last":0.71899999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.747,"15min":0.71399999999999997},"min":{"1min":0.58799999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91100000000000003,"5min":1.01,"15min":1.01},"last":0.753},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.78900000000000003,"15min":0.75},"min":{"1min":0.627,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0640000000000001,"5min":1.107,"15min":1.107},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri Nov 17 00:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89500000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.88900000000000001,"15min":0.89500000000000002},"min":{"1min":0.79400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80100000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6812,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738340,"statfs":{"total":95995035648,"available":95988060160,"internally_reserved":0,"allocated":1003520,"data_stored":509199,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48899999999999999,"15min":0.42699999999999999},"min":{"1min":0.34799999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":1.484,"15min":1.484},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35499999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.61399999999999999},"min":{"1min":0.52500000000000002,"5min":0.42899999999999999,"15min":0.36099999999999999},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1399999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.64700000000000002},"min":{"1min":0.46200000000000002,"5min":0.45300000000000001,"15min":0.40100000000000002},"max":{"1min":0.86499999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85899999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.90800000000000003,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Nov 17 00:07:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73299999999999998,"15min":0.68100000000000005},"min":{"1min":0.49099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.90200000000000002,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.68400000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95899999999999996,"5min":1.359,"15min":1.359},"last":0.873}]},{"osd":7,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.76200000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.78600000000000003,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.66600000000000004}]},{"osd":8,"last update":"Fri Nov 17 00:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.87,"15min":0.85499999999999998},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Nov 17 00:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.92000000000000004},"min":{"1min":0.55200000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.96499999999999997,"15min":1.0209999999999999},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Nov 17 00:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.95599999999999996,"15min":1.0009999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4850000000000001,"5min":1.496,"15min":1.496},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.96499999999999997,"15min":1.0129999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.268,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Fri Nov 17 00:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.89100000000000001,"15min":0.90400000000000003},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":2.101,"15min":2.101},"last":0.748},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.177,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.75800000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":1007616,"data_stored":513544,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.46500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.35999999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.751,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.435,"15min":0.373},"min":{"1min":0.27600000000000002,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":0.875,"15min":0.875},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.56599999999999995},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Nov 17 00:08:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.55600000000000005},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.38300000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79600000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 17 00:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.626,"15min":0.57299999999999995},"min":{"1min":0.47199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63800000000000001,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 17 00:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67600000000000005,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88400000000000001,"5min":1,"15min":1},"last":0.37},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91200000000000003,"5min":0.999,"15min":0.999},"last":0.47099999999999997}]},{"osd":8,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65500000000000003,"15min":0.63400000000000001},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.74099999999999999,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0349999999999999,"5min":1.105,"15min":1.105},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Nov 17 00:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79000000000000004,"15min":0.80400000000000005},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":0.93799999999999994,"5min":1.081,"15min":1.081},"last":0.59899999999999998}]},{"osd":10,"last update":"Fri Nov 17 00:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.441},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.39800000000000002}]},{"osd":11,"last update":"Fri Nov 17 00:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82899999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.03,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.752},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]}]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6872,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738280,"statfs":{"total":95995035648,"available":95987998720,"internally_reserved":0,"allocated":1064960,"data_stored":1104072,"data_compressed":59369,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.62,"15min":0.56699999999999995},"min":{"1min":0.44,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":6.8310000000000004,"5min":6.8310000000000004,"15min":6.8310000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.60599999999999998,"15min":0.56299999999999994},"min":{"1min":0.496,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.436,"15min":0.39500000000000002},"min":{"1min":0.38700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64700000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.44,"15min":0.40699999999999997},"min":{"1min":0.38900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60799999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.45300000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.54100000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Nov 17 00:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60999999999999999,"15min":0.58099999999999996},"min":{"1min":0.50800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.65100000000000002,"15min":0.59099999999999997},"min":{"1min":0.51700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 17 00:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67600000000000005,"15min":0.66500000000000004},"min":{"1min":0.56499999999999995,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85299999999999998,"5min":1.05,"15min":1.05},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84799999999999998,"5min":1.046,"15min":1.046},"last":0.48299999999999998}]},{"osd":7,"last update":"Fri Nov 17 00:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.625},"min":{"1min":0.60799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52500000000000002}]},{"osd":8,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65000000000000002,"15min":0.61799999999999999},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85799999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Nov 17 00:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.60899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":1.022,"15min":1.022},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75900000000000001,"15min":0.74199999999999999},"min":{"1min":0.59799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71199999999999997}]},{"osd":10,"last update":"Fri Nov 17 00:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78700000000000003,"15min":0.77000000000000002},"min":{"1min":0.66000000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.073,"15min":1.073},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.92500000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34186,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-17T00:08:40.169 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 -- 172.21.15.6:0/3459580742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a681002d0 msgr2=0x7f1a681006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/3459580742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a681002d0 0x7f1a681006e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009990 tx=0x7f1a58009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 -- 172.21.15.6:0/3459580742 shutdown_connections 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/3459580742 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a681002d0 0x7f1a681006e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/3459580742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68101e50 0x7f1a68108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/3459580742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681014c0 0x7f1a68101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 -- 172.21.15.6:0/3459580742 >> 172.21.15.6:0/3459580742 conn(0x7f1a680fb8b0 msgr2=0x7f1a680fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:40.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 -- 172.21.15.6:0/3459580742 shutdown_connections 2023-11-17T00:08:40.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.391+0000 7f1a6cbff700 1 -- 172.21.15.6:0/3459580742 wait complete. 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.392+0000 7f1a6cbff700 1 Processor -- start 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.392+0000 7f1a6cbff700 1 -- start start 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681002d0 0x7f1a68198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 0x7f1a681987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6819fe00 con 0x7f1a68101e50 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6819ff40 con 0x7f1a681014c0 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681a00d0 con 0x7f1a681002d0 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a6659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681002d0 0x7f1a68198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60476/0 (socket says 172.21.15.6:60476) 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 -- 172.21.15.6:0/2520715326 learned_addr learned my addr 172.21.15.6:0/2520715326 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a5ffff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 0x7f1a681987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 -- 172.21.15.6:0/2520715326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681002d0 msgr2=0x7f1a68198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681002d0 0x7f1a68198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 -- 172.21.15.6:0/2520715326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 msgr2=0x7f1a681987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 0x7f1a681987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a66d9d700 1 -- 172.21.15.6:0/2520715326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580095f0 con 0x7f1a68101e50 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.393+0000 7f1a5ffff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 0x7f1a681987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a66d9d700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1a60007ee0 tx=0x7f1a6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60017070 con 0x7f1a68101e50 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6000f040 con 0x7f1a68101e50 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60005740 con 0x7f1a68101e50 2023-11-17T00:08:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6810bca0 con 0x7f1a68101e50 2023-11-17T00:08:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.394+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6810bb10 con 0x7f1a68101e50 2023-11-17T00:08:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.395+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68192a00 con 0x7f1a68101e50 2023-11-17T00:08:40.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.397+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1a6000dc60 con 0x7f1a68101e50 2023-11-17T00:08:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.398+0000 7f1a5f7fe700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 0x7f1a540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.399+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1a600cc4b0 con 0x7f1a68101e50 2023-11-17T00:08:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.399+0000 7f1a6659c700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 0x7f1a540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.399+0000 7f1a6659c700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 0x7f1a540a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800b2e0 tx=0x7f1a58005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:40.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.403+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a60095830 con 0x7f1a68101e50 2023-11-17T00:08:40.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.616+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a6802cfe0 con 0x7f1a68101e50 2023-11-17T00:08:40.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.618+0000 7f1a5f7fe700 1 -- 172.21.15.6:0/2520715326 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7f1a60098e50 con 0x7f1a68101e50 2023-11-17T00:08:40.619 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:40.619 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 msgr2=0x7f1a540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 0x7f1a540a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800b2e0 tx=0x7f1a58005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 msgr2=0x7f1a6819f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1a60007ee0 tx=0x7f1a6000d3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 shutdown_connections 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f1a540a1150 0x7f1a540a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a68101e50 0x7f1a6819f830 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a681002d0 0x7f1a68198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 --2- 172.21.15.6:0/2520715326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a681014c0 0x7f1a681987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 >> 172.21.15.6:0/2520715326 conn(0x7f1a680fb8b0 msgr2=0x7f1a68106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:40.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 shutdown_connections 2023-11-17T00:08:40.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.620+0000 7f1a6cbff700 1 -- 172.21.15.6:0/2520715326 wait complete. 2023-11-17T00:08:40.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:08:40.641 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:08:40.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:40 smithi087 ceph-mon[131182]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:40 smithi087 ceph-mon[131182]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:40 smithi087 ceph-mon[131182]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 -- 172.21.15.6:0/3171239765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8101480 msgr2=0x7efbe81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 --2- 172.21.15.6:0/3171239765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8101480 0x7efbe81018d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efbd8009990 tx=0x7efbd8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 -- 172.21.15.6:0/3171239765 shutdown_connections 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 --2- 172.21.15.6:0/3171239765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101e10 0x7efbe81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 --2- 172.21.15.6:0/3171239765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8100290 0x7efbe81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 --2- 172.21.15.6:0/3171239765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8101480 0x7efbe81018d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 -- 172.21.15.6:0/3171239765 >> 172.21.15.6:0/3171239765 conn(0x7efbe80fb870 msgr2=0x7efbe80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.850+0000 7efbefbff700 1 -- 172.21.15.6:0/3171239765 shutdown_connections 2023-11-17T00:08:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.851+0000 7efbefbff700 1 -- 172.21.15.6:0/3171239765 wait complete. 2023-11-17T00:08:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 Processor -- start 2023-11-17T00:08:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 -- start start 2023-11-17T00:08:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101480 0x7efbe8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8101e10 0x7efbe819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe819fd60 con 0x7efbe8100290 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe819fea0 con 0x7efbe8101480 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbefbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe81a0030 con 0x7efbe8101e10 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbed99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbed99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60500/0 (socket says 172.21.15.6:60500) 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbed99b700 1 -- 172.21.15.6:0/2268701861 learned_addr learned my addr 172.21.15.6:0/2268701861 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.852+0000 7efbed19a700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101480 0x7efbe8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbee19c700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8101e10 0x7efbe819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 -- 172.21.15.6:0/2268701861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8101e10 msgr2=0x7efbe819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8101e10 0x7efbe819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 -- 172.21.15.6:0/2268701861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101480 msgr2=0x7efbe8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101480 0x7efbe8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 -- 172.21.15.6:0/2268701861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbd80095f0 con 0x7efbe8100290 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbed99b700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e970 tx=0x7efbe400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4009790 con 0x7efbe8100290 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbe810bc50 con 0x7efbe8100290 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbe400eec0 con 0x7efbe8100290 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4003cf0 con 0x7efbe8100290 2023-11-17T00:08:40.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.853+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbe810bac0 con 0x7efbe8100290 2023-11-17T00:08:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.854+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbe81a0780 con 0x7efbe8100290 2023-11-17T00:08:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.856+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efbe400ccd0 con 0x7efbe8100290 2023-11-17T00:08:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.858+0000 7efbdeffd700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 0x7efbd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.858+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efbe4014070 con 0x7efbe8100290 2023-11-17T00:08:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.858+0000 7efbed19a700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 0x7efbd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.858+0000 7efbed19a700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 0x7efbd40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efbd800b2e0 tx=0x7efbd8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:40.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:40.861+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efbe40962b0 con 0x7efbe8100290 2023-11-17T00:08:40.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:40 smithi186 ceph-mon[131487]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:40 smithi186 ceph-mon[131487]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:40 smithi186 ceph-mon[131487]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:40 smithi006 ceph-mon[97840]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:40 smithi006 ceph-mon[97840]: from='client.25345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:40 smithi006 ceph-mon[97840]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.069+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efbe81a0d30 con 0x7efbe8100290 2023-11-17T00:08:41.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.070+0000 7efbdeffd700 1 -- 172.21.15.6:0/2268701861 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7efbe40998d0 con 0x7efbe8100290 2023-11-17T00:08:41.071 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 msgr2=0x7efbd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 0x7efbd40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efbd800b2e0 tx=0x7efbd8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 msgr2=0x7efbe81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e970 tx=0x7efbe400ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 shutdown_connections 2023-11-17T00:08:41.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7efbd40a1150 0x7efbd40a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efbe8100290 0x7efbe81981e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efbe8101e10 0x7efbe819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 --2- 172.21.15.6:0/2268701861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efbe8101480 0x7efbe8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 >> 172.21.15.6:0/2268701861 conn(0x7efbe80fb870 msgr2=0x7efbe80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 shutdown_connections 2023-11-17T00:08:41.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.072+0000 7efbefbff700 1 -- 172.21.15.6:0/2268701861 wait complete. 2023-11-17T00:08:41.092 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 -- 172.21.15.6:0/536695230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c8074b30 msgr2=0x7f07c8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 --2- 172.21.15.6:0/536695230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c8074b30 0x7f07c8072f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f07c40098f0 tx=0x7f07c4009c00 comp rx=0 tx=0).stop 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 -- 172.21.15.6:0/536695230 shutdown_connections 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 --2- 172.21.15.6:0/536695230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c810c860 0x7f07c810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 --2- 172.21.15.6:0/536695230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c8074b30 0x7f07c8072f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 --2- 172.21.15.6:0/536695230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 0x7f07c80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 -- 172.21.15.6:0/536695230 >> 172.21.15.6:0/536695230 conn(0x7f07c80fb870 msgr2=0x7f07c80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 -- 172.21.15.6:0/536695230 shutdown_connections 2023-11-17T00:08:41.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.301+0000 7f07d05ff700 1 -- 172.21.15.6:0/536695230 wait complete. 2023-11-17T00:08:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 Processor -- start 2023-11-17T00:08:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 -- start start 2023-11-17T00:08:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 0x7f07c819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 0x7f07c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 0x7f07c81a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a4240 con 0x7f07c8074b30 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a4380 con 0x7f07c8073550 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.302+0000 7f07d05ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a4510 con 0x7f07c810c860 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 0x7f07c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07ceb9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 0x7f07c81a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07ce39b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 0x7f07c819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07ceb9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 0x7f07c81a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49292/0 (socket says 172.21.15.6:49292) 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07ceb9c700 1 -- 172.21.15.6:0/1485449647 learned_addr learned my addr 172.21.15.6:0/1485449647 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 -- 172.21.15.6:0/1485449647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 msgr2=0x7f07c81a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 0x7f07c81a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 -- 172.21.15.6:0/1485449647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 msgr2=0x7f07c819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 0x7f07c819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 -- 172.21.15.6:0/1485449647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4009520 con 0x7f07c8074b30 2023-11-17T00:08:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07cdb9a700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 0x7f07c819cc00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9c0 tx=0x7f07b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.303+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004220 con 0x7f07c8074b30 2023-11-17T00:08:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.304+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b8004380 con 0x7f07c8074b30 2023-11-17T00:08:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.304+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003be0 con 0x7f07c8074b30 2023-11-17T00:08:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.304+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c8112250 con 0x7f07c8074b30 2023-11-17T00:08:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.304+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c81120c0 con 0x7f07c8074b30 2023-11-17T00:08:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.305+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c81023b0 con 0x7f07c8074b30 2023-11-17T00:08:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.307+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f07b8003d40 con 0x7f07c8074b30 2023-11-17T00:08:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.308+0000 7f07bf7fe700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 0x7f07b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.308+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f07b8014070 con 0x7f07c8074b30 2023-11-17T00:08:41.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.308+0000 7f07ce39b700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 0x7f07b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.309+0000 7f07ce39b700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 0x7f07b40a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:41.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.310+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07b8096490 con 0x7f07c8074b30 2023-11-17T00:08:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.518+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07c8061700 con 0x7f07c8074b30 2023-11-17T00:08:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.518+0000 7f07bf7fe700 1 -- 172.21.15.6:0/1485449647 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07b801f340 con 0x7f07c8074b30 2023-11-17T00:08:41.523 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:41.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 msgr2=0x7f07b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 0x7f07b40a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009fd0 tx=0x7f07c4019040 comp rx=0 tx=0).stop 2023-11-17T00:08:41.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 msgr2=0x7f07c819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 0x7f07c819cc00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07b800e9c0 tx=0x7f07b800ecd0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 shutdown_connections 2023-11-17T00:08:41.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07b40a1150 0x7f07b40a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07c810c860 0x7f07c81a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07c8073550 0x7f07c819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 --2- 172.21.15.6:0/1485449647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07c8074b30 0x7f07c819cc00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 >> 172.21.15.6:0/1485449647 conn(0x7f07c80fb870 msgr2=0x7f07c80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:41.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 shutdown_connections 2023-11-17T00:08:41.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.521+0000 7f07d05ff700 1 -- 172.21.15.6:0/1485449647 wait complete. 2023-11-17T00:08:41.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 -- 172.21.15.6:0/10664010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0101480 msgr2=0x7fdbd01018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/10664010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0101480 0x7fdbd01018d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009990 tx=0x7fdbb8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 -- 172.21.15.6:0/10664010 shutdown_connections 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/10664010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101e10 0x7fdbd01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/10664010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0100290 0x7fdbd01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/10664010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0101480 0x7fdbd01018d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 -- 172.21.15.6:0/10664010 >> 172.21.15.6:0/10664010 conn(0x7fdbd00fb870 msgr2=0x7fdbd00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 -- 172.21.15.6:0/10664010 shutdown_connections 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.747+0000 7fdbd4fff700 1 -- 172.21.15.6:0/10664010 wait complete. 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 Processor -- start 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 -- start start 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101480 0x7fdbd0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0101e10 0x7fdbd019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd019fd60 con 0x7fdbd0100290 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd019fea0 con 0x7fdbd0101480 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.748+0000 7fdbd4fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01a0030 con 0x7fdbd0101e10 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60522/0 (socket says 172.21.15.6:60522) 2023-11-17T00:08:41.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 -- 172.21.15.6:0/2578548939 learned_addr learned my addr 172.21.15.6:0/2578548939 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:41.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbc7fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101480 0x7fdbd0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbced9d700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0101e10 0x7fdbd019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 -- 172.21.15.6:0/2578548939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0101e10 msgr2=0x7fdbd019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0101e10 0x7fdbd019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 -- 172.21.15.6:0/2578548939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101480 msgr2=0x7fdbd0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101480 0x7fdbd0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 -- 172.21.15.6:0/2578548939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80095f0 con 0x7fdbd0100290 2023-11-17T00:08:41.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.749+0000 7fdbce59c700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d980 tx=0x7fdbc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:41.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.750+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0004020 con 0x7fdbd0100290 2023-11-17T00:08:41.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.750+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd010bc50 con 0x7fdbd0100290 2023-11-17T00:08:41.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.750+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc000f690 con 0x7fdbd0100290 2023-11-17T00:08:41.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.750+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd010bac0 con 0x7fdbd0100290 2023-11-17T00:08:41.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.750+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003a00 con 0x7fdbd0100290 2023-11-17T00:08:41.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.751+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd01a07d0 con 0x7fdbd0100290 2023-11-17T00:08:41.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.753+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdbc000fb00 con 0x7fdbd0100290 2023-11-17T00:08:41.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.754+0000 7fdbc77fe700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 0x7fdbbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:41.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.754+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdbc00cc6b0 con 0x7fdbd0100290 2023-11-17T00:08:41.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.754+0000 7fdbc7fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 0x7fdbbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:41.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.755+0000 7fdbc7fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 0x7fdbbc0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800b5c0 tx=0x7fdbb8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:41.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.758+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbc0095a30 con 0x7fdbd0100290 2023-11-17T00:08:41.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:41 smithi087 ceph-mon[131182]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:41.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:41 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2520715326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:41.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:41 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2268701861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:41.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:41 smithi186 ceph-mon[131487]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:41.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:41 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2520715326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:41.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:41 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2268701861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:41.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.969+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbd01a0d80 con 0x7fdbd0100290 2023-11-17T00:08:41.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.969+0000 7fdbc77fe700 1 -- 172.21.15.6:0/2578548939 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbc0099050 con 0x7fdbd0100290 2023-11-17T00:08:41.970 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 msgr2=0x7fdbbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 0x7fdbbc0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800b5c0 tx=0x7fdbb8005fd0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 msgr2=0x7fdbd01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000d980 tx=0x7fdbc000dc90 comp rx=0 tx=0).stop 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 shutdown_connections 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fdbbc0a1150 0x7fdbbc0a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdbd0100290 0x7fdbd01981e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbd0101e10 0x7fdbd019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 --2- 172.21.15.6:0/2578548939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbd0101480 0x7fdbd0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:41.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.971+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 >> 172.21.15.6:0/2578548939 conn(0x7fdbd00fb870 msgr2=0x7fdbd00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:41.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.972+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 shutdown_connections 2023-11-17T00:08:41.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:41.972+0000 7fdbd4fff700 1 -- 172.21.15.6:0/2578548939 wait complete. 2023-11-17T00:08:41.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:41 smithi006 ceph-mon[97840]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-17T00:08:41.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2520715326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:41.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:41 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2268701861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:08:41.992 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:08:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 -- 172.21.15.6:0/3735816213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 msgr2=0x7f24b80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 --2- 172.21.15.6:0/3735816213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b80739c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009990 tx=0x7f24a8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 -- 172.21.15.6:0/3735816213 shutdown_connections 2023-11-17T00:08:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 --2- 172.21.15.6:0/3735816213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b810c860 0x7f24b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 --2- 172.21.15.6:0/3735816213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b8074b30 0x7f24b8072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 --2- 172.21.15.6:0/3735816213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b80739c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 -- 172.21.15.6:0/3735816213 >> 172.21.15.6:0/3735816213 conn(0x7f24b80fb870 msgr2=0x7f24b80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 -- 172.21.15.6:0/3735816213 shutdown_connections 2023-11-17T00:08:42.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.200+0000 7f24bfbff700 1 -- 172.21.15.6:0/3735816213 wait complete. 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.201+0000 7f24bfbff700 1 Processor -- start 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 -- start start 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8074b30 0x7f24b819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b810c860 0x7f24b81a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a41a0 con 0x7f24b8073550 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a42e0 con 0x7f24b8074b30 2023-11-17T00:08:42.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bfbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81a4470 con 0x7f24b810c860 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bd99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bd99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60538/0 (socket says 172.21.15.6:60538) 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bd99b700 1 -- 172.21.15.6:0/2042716613 learned_addr learned my addr 172.21.15.6:0/2042716613 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.202+0000 7f24bd19a700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8074b30 0x7f24b819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24be19c700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b810c860 0x7f24b81a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 -- 172.21.15.6:0/2042716613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b810c860 msgr2=0x7f24b81a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b810c860 0x7f24b81a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 -- 172.21.15.6:0/2042716613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8074b30 msgr2=0x7f24b819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8074b30 0x7f24b819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 -- 172.21.15.6:0/2042716613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80095f0 con 0x7f24b8073550 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bd99b700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24b400e970 tx=0x7f24b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4009790 con 0x7f24b8073550 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b400eec0 con 0x7f24b8073550 2023-11-17T00:08:42.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b8112270 con 0x7f24b8073550 2023-11-17T00:08:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4005590 con 0x7f24b8073550 2023-11-17T00:08:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.203+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81120e0 con 0x7f24b8073550 2023-11-17T00:08:42.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.204+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b81023b0 con 0x7f24b8073550 2023-11-17T00:08:42.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.206+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f24b400cbd0 con 0x7f24b8073550 2023-11-17T00:08:42.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.208+0000 7f24aeffd700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 0x7f24a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.208+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f24b4014070 con 0x7f24b8073550 2023-11-17T00:08:42.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.208+0000 7f24bd19a700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 0x7f24a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.209+0000 7f24bd19a700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 0x7f24a40a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b2e0 tx=0x7f24a8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:42.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.212+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f24b40d2020 con 0x7f24b8073550 2023-11-17T00:08:42.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.422+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f24b8061700 con 0x7f24b8073550 2023-11-17T00:08:42.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.422+0000 7f24aeffd700 1 -- 172.21.15.6:0/2042716613 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f24b4099ce0 con 0x7f24b8073550 2023-11-17T00:08:42.425 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:08:42.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 msgr2=0x7f24a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 0x7f24a40a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b2e0 tx=0x7f24a8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 msgr2=0x7f24b819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24b400e970 tx=0x7f24b400ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 shutdown_connections 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f24a40a11d0 0x7f24a40a3680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24b8073550 0x7f24b819c620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24b810c860 0x7f24b81a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 --2- 172.21.15.6:0/2042716613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24b8074b30 0x7f24b819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 >> 172.21.15.6:0/2042716613 conn(0x7f24b80fb870 msgr2=0x7f24b80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:42.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 shutdown_connections 2023-11-17T00:08:42.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.425+0000 7f24bfbff700 1 -- 172.21.15.6:0/2042716613 wait complete. 2023-11-17T00:08:42.446 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:08:42.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.655+0000 7fa0bcbff700 1 -- 172.21.15.6:0/308588730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b8101e50 msgr2=0x7fa0b8108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.655+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/308588730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b8101e50 0x7fa0b8108710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000b0a0 tx=0x7fa0b000b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 -- 172.21.15.6:0/308588730 shutdown_connections 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/308588730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b8101e50 0x7fa0b8108710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/308588730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b81002d0 0x7fa0b81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/308588730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81014c0 0x7fa0b8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 -- 172.21.15.6:0/308588730 >> 172.21.15.6:0/308588730 conn(0x7fa0b80fb8b0 msgr2=0x7fa0b80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 -- 172.21.15.6:0/308588730 shutdown_connections 2023-11-17T00:08:42.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.656+0000 7fa0bcbff700 1 -- 172.21.15.6:0/308588730 wait complete. 2023-11-17T00:08:42.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.657+0000 7fa0bcbff700 1 Processor -- start 2023-11-17T00:08:42.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.657+0000 7fa0bcbff700 1 -- start start 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 0x7fa0b819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b819dcf0 con 0x7fa0b81014c0 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b819de30 con 0x7fa0b8101e50 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0bcbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b819dfc0 con 0x7fa0b81002d0 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0affff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60564/0 (socket says 172.21.15.6:60564) 2023-11-17T00:08:42.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0b659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49354/0 (socket says 172.21.15.6:49354) 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0affff700 1 -- 172.21.15.6:0/2151350825 learned_addr learned my addr 172.21.15.6:0/2151350825 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.658+0000 7fa0b6d9d700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 0x7fa0b819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 -- 172.21.15.6:0/2151350825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 msgr2=0x7fa0b819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 0x7fa0b819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 -- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 msgr2=0x7fa0b81966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 -- 172.21.15.6:0/2151350825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0009d00 con 0x7fa0b81002d0 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0affff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b6d9d700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 0x7fa0b819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0b659c700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9c0 tx=0x7fa0a800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:42.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8004220 con 0x7fa0b81002d0 2023-11-17T00:08:42.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a8004380 con 0x7fa0b81002d0 2023-11-17T00:08:42.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b810bcc0 con 0x7fa0b81002d0 2023-11-17T00:08:42.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8003be0 con 0x7fa0b81002d0 2023-11-17T00:08:42.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.659+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b810bb30 con 0x7fa0b81002d0 2023-11-17T00:08:42.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.660+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b81077d0 con 0x7fa0b81002d0 2023-11-17T00:08:42.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.662+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa0a8003d40 con 0x7fa0b81002d0 2023-11-17T00:08:42.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.664+0000 7fa0af7fe700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 0x7fa0a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:42.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.664+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa0a8014070 con 0x7fa0b81002d0 2023-11-17T00:08:42.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.664+0000 7fa0affff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 0x7fa0a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:42.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.664+0000 7fa0affff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 0x7fa0a40a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0011590 tx=0x7fa0a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:42.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.666+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0a80968d0 con 0x7fa0b81002d0 2023-11-17T00:08:42.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1485449647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.727 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:42 smithi006 ceph-mon[97840]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2023-11-17T00:08:42.729 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2578548939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.731 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:42 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2042716613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1485449647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:42 smithi087 ceph-mon[131182]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2023-11-17T00:08:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2578548939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:42 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2042716613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.871 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1485449647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:42 smithi186 ceph-mon[131487]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2023-11-17T00:08:42.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2578548939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.872 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:42 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2042716613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:08:42.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.916+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0b8061700 con 0x7fa0b81002d0 2023-11-17T00:08:42.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.917+0000 7fa0af7fe700 1 -- 172.21.15.6:0/2151350825 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7fa0a801f340 con 0x7fa0b81002d0 2023-11-17T00:08:42.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:42.918 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:08:42.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 msgr2=0x7fa0a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 0x7fa0a40a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0011590 tx=0x7fa0a0009260 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 msgr2=0x7fa0b8196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9c0 tx=0x7fa0a800ecd0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 shutdown_connections 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fa0a40a1150 0x7fa0a40a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0b8101e50 0x7fa0b819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0b81002d0 0x7fa0b8196170 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 --2- 172.21.15.6:0/2151350825 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa0b81014c0 0x7fa0b81966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 >> 172.21.15.6:0/2151350825 conn(0x7fa0b80fb8b0 msgr2=0x7fa0b8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 shutdown_connections 2023-11-17T00:08:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:42.920+0000 7fa0bcbff700 1 -- 172.21.15.6:0/2151350825 wait complete. 2023-11-17T00:08:42.922 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:08:42.940 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 -- 172.21.15.6:0/3925637720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 msgr2=0x7f07e01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 --2- 172.21.15.6:0/3925637720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e01006e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f07d00098f0 tx=0x7f07d0009c00 comp rx=0 tx=0).stop 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 -- 172.21.15.6:0/3925637720 shutdown_connections 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 --2- 172.21.15.6:0/3925637720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e01006e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 --2- 172.21.15.6:0/3925637720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0101e50 0x7f07e0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 --2- 172.21.15.6:0/3925637720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e01014c0 0x7f07e0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 -- 172.21.15.6:0/3925637720 >> 172.21.15.6:0/3925637720 conn(0x7f07e00fb8b0 msgr2=0x7f07e00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 -- 172.21.15.6:0/3925637720 shutdown_connections 2023-11-17T00:08:43.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.146+0000 7f07e49ff700 1 -- 172.21.15.6:0/3925637720 wait complete. 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.147+0000 7f07e49ff700 1 Processor -- start 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.147+0000 7f07e49ff700 1 -- start start 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e01014c0 0x7f07e0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e019fe80 con 0x7f07e01002d0 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e019ffc0 con 0x7f07e01014c0 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07e49ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01a0150 con 0x7f07e0101e50 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07de59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49380/0 (socket says 172.21.15.6:49380) 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 -- 172.21.15.6:0/2569964673 learned_addr learned my addr 172.21.15.6:0/2569964673 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ddd9b700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e01014c0 0x7f07e0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 -- 172.21.15.6:0/2569964673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e01014c0 msgr2=0x7f07e0198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e01014c0 0x7f07e0198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 -- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 msgr2=0x7f07e0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07ded9d700 1 -- 172.21.15.6:0/2569964673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d0009520 con 0x7f07e0101e50 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.148+0000 7f07de59c700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07ded9d700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c110 tx=0x7f07d800c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8017070 con 0x7f07e0101e50 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e010bca0 con 0x7f07e0101e50 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d800e050 con 0x7f07e0101e50 2023-11-17T00:08:43.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80078f0 con 0x7f07e0101e50 2023-11-17T00:08:43.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.149+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e010bb10 con 0x7f07e0101e50 2023-11-17T00:08:43.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.150+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0192a00 con 0x7f07e0101e50 2023-11-17T00:08:43.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.152+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f07d8007a50 con 0x7f07e0101e50 2023-11-17T00:08:43.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.154+0000 7f07cf7fe700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 0x7f07c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.154+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f07d80cc850 con 0x7f07e0101e50 2023-11-17T00:08:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.154+0000 7f07de59c700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 0x7f07c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.154+0000 7f07de59c700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 0x7f07c80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009fd0 tx=0x7f07d0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.155+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07d8095bd0 con 0x7f07e0101e50 2023-11-17T00:08:43.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.366+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07e002cfe0 con 0x7f07e0101e50 2023-11-17T00:08:43.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.368+0000 7f07cf7fe700 1 -- 172.21.15.6:0/2569964673 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24477 (secure 0 0 0) 0x7f07d804dfb0 con 0x7f07e0101e50 2023-11-17T00:08:43.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:08:43.369 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":92,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:08:03.838535+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 msgr2=0x7f07c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 0x7f07c80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009fd0 tx=0x7f07d0019040 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 msgr2=0x7f07e019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c110 tx=0x7f07d800c420 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 shutdown_connections 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f07c80a1150 0x7f07c80a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07e01002d0 0x7f07e0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101e50 0x7f07e019f8b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 --2- 172.21.15.6:0/2569964673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e01014c0 0x7f07e0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 >> 172.21.15.6:0/2569964673 conn(0x7f07e00fb8b0 msgr2=0x7f07e0106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 shutdown_connections 2023-11-17T00:08:43.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.370+0000 7f07e49ff700 1 -- 172.21.15.6:0/2569964673 wait complete. 2023-11-17T00:08:43.390 DEBUG:tasks.fwd_scrub:Yielding 2023-11-17T00:08:43.390 INFO:teuthology.run_tasks:Running task exec... 2023-11-17T00:08:43.404 INFO:teuthology.task.exec:Executing custom commands... 2023-11-17T00:08:43.404 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:08:43.404 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-11-17T00:08:43.404 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-11-17T00:08:43.404 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:08:43.405 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 -- 172.21.15.6:0/1505668951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064074a90 msgr2=0x7f4064072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 --2- 172.21.15.6:0/1505668951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064074a90 0x7f4064072f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4060009990 tx=0x7f4060009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 -- 172.21.15.6:0/1505668951 shutdown_connections 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 --2- 172.21.15.6:0/1505668951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064103ca0 0x7f406410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 --2- 172.21.15.6:0/1505668951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064074a90 0x7f4064072f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 --2- 172.21.15.6:0/1505668951 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f4064073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 -- 172.21.15.6:0/1505668951 >> 172.21.15.6:0/1505668951 conn(0x7f40640fb810 msgr2=0x7f40640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 -- 172.21.15.6:0/1505668951 shutdown_connections 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.607+0000 7f406c3ff700 1 -- 172.21.15.6:0/1505668951 wait complete. 2023-11-17T00:08:43.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.608+0000 7f406c3ff700 1 Processor -- start 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 -- start start 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f406419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 0x7f406419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064103ca0 0x7f40641a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641a4170 con 0x7f40640734c0 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641a42b0 con 0x7f4064074a90 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641a4440 con 0x7f4064103ca0 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f406419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406999a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 0x7f406419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406a19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f406419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60602/0 (socket says 172.21.15.6:60602) 2023-11-17T00:08:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406a19b700 1 -- 172.21.15.6:0/2852292195 learned_addr learned my addr 172.21.15.6:0/2852292195 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.609+0000 7f406a99c700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064103ca0 0x7f40641a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 -- 172.21.15.6:0/2852292195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064103ca0 msgr2=0x7f40641a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064103ca0 0x7f40641a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 -- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 msgr2=0x7f406419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f406419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 -- 172.21.15.6:0/2852292195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600095f0 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406999a700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 0x7f406419cb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f405400e9f0 tx=0x7f405400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400eee0 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40641121f0 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405400cbd0 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054018680 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.610+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4064112060 con 0x7f4064074a90 2023-11-17T00:08:43.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.611+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40641023f0 con 0x7f4064074a90 2023-11-17T00:08:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.613+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f405400cd40 con 0x7f4064074a90 2023-11-17T00:08:43.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.615+0000 7f405b7fe700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 0x7f40500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.615+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4054014070 con 0x7f4064074a90 2023-11-17T00:08:43.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.615+0000 7f406a19b700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 0x7f40500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.616+0000 7f406a19b700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 0x7f40500a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f406000b2e0 tx=0x7f4060005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.617+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4054096260 con 0x7f4064074a90 2023-11-17T00:08:43.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.620+0000 7fda3199d700 1 -- 172.21.15.6:0/2204596905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101250 msgr2=0x7fda2c1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.620+0000 7fda3199d700 1 --2- 172.21.15.6:0/2204596905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101250 0x7fda2c1016a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda180098f0 tx=0x7fda18009c00 comp rx=0 tx=0).stop 2023-11-17T00:08:43.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 -- 172.21.15.6:0/2204596905 shutdown_connections 2023-11-17T00:08:43.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 --2- 172.21.15.6:0/2204596905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c100060 0x7fda2c100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 --2- 172.21.15.6:0/2204596905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c101be0 0x7fda2c1084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 --2- 172.21.15.6:0/2204596905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101250 0x7fda2c1016a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 -- 172.21.15.6:0/2204596905 >> 172.21.15.6:0/2204596905 conn(0x7fda2c0fb6a0 msgr2=0x7fda2c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 -- 172.21.15.6:0/2204596905 shutdown_connections 2023-11-17T00:08:43.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.621+0000 7fda3199d700 1 -- 172.21.15.6:0/2204596905 wait complete. 2023-11-17T00:08:43.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 Processor -- start 2023-11-17T00:08:43.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 -- start start 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 0x7fda2c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101be0 0x7fda2c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19fb60 con 0x7fda2c101250 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19fca0 con 0x7fda2c100060 2023-11-17T00:08:43.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda3199d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c19fe30 con 0x7fda2c101be0 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda3099b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 0x7fda2c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.622+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60632/0 (socket says 172.21.15.6:60632) 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda3119c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101be0 0x7fda2c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 -- 172.21.15.6:0/3476204805 learned_addr learned my addr 172.21.15.6:0/3476204805 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 -- 172.21.15.6:0/3476204805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101be0 msgr2=0x7fda2c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101be0 0x7fda2c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 -- 172.21.15.6:0/3476204805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 msgr2=0x7fda2c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 0x7fda2c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 -- 172.21.15.6:0/3476204805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda18009520 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda3099b700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 0x7fda2c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.623+0000 7fda2bfff700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fda18009fd0 tx=0x7fda1800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.624+0000 7fda29ffb700 1 -- 172.21.15.6:0/3476204805 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1801c070 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.624+0000 7fda29ffb700 1 -- 172.21.15.6:0/3476204805 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1800f7d0 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.624+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c10b9b0 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.624+0000 7fda29ffb700 1 -- 172.21.15.6:0/3476204805 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18004ab0 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.624+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c10b820 con 0x7fda2c101250 2023-11-17T00:08:43.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.627+0000 7fda29ffb700 1 -- 172.21.15.6:0/3476204805 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fda18020070 con 0x7fda2c101250 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.628+0000 7fda465fdb80 1 Processor -- start 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.628+0000 7fda465fdb80 1 -- start start 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 0x559a08ca0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca1330 0x559a08ca17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda29ffb700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08ba50e0 con 0x559a08ca1330 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08bf7920 con 0x559a08c9ff10 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda465fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08bf0db0 con 0x559a08ca0860 2023-11-17T00:08:43.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda29ffb700 1 -- 172.21.15.6:0/3476204805 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda180cd8e0 con 0x7fda2c101250 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda3119c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 0x559a08ca0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda3119c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 0x559a08ca0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33066/0 (socket says 172.21.15.6:33066) 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda3119c700 1 -- 172.21.15.6:0/3866768339 learned_addr learned my addr 172.21.15.6:0/3866768339 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda3099b700 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda3119c700 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca1330 0x559a08ca17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.629+0000 7fda2bfff700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 -- 172.21.15.6:0/3866768339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 msgr2=0x559a08ca0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 0x559a08ca0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 -- 172.21.15.6:0/3866768339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca1330 msgr2=0x559a08ca17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca1330 0x559a08ca17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 -- 172.21.15.6:0/3866768339 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a08a726b0 con 0x559a08ca0860 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda3099b700 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca0da0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda20005fd0 tx=0x7fda200117a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4af04ecaa1f27fa server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda2bfff700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 0x7fda1c0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda180058b0 tx=0x7fda18009c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda16ffd700 1 -- 172.21.15.6:0/3866768339 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20011310 con 0x559a08ca0860 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda16ffd700 1 -- 172.21.15.6:0/3866768339 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda200044f0 con 0x559a08ca0860 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.630+0000 7fda16ffd700 1 -- 172.21.15.6:0/3866768339 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001a6a0 con 0x559a08ca0860 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 -- 172.21.15.6:0/3866768339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 msgr2=0x559a08ca0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca0da0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda20005fd0 tx=0x7fda200117a0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 -- 172.21.15.6:0/3866768339 shutdown_connections 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca1330 0x559a08ca17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c9ff10 0x559a08ca0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 --2- 172.21.15.6:0/3866768339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca0da0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 -- 172.21.15.6:0/3866768339 >> 172.21.15.6:0/3866768339 conn(0x559a08c95630 msgr2=0x559a08c9a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 -- 172.21.15.6:0/3866768339 shutdown_connections 2023-11-17T00:08:43.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.631+0000 7fda465fdb80 1 -- 172.21.15.6:0/3866768339 wait complete. 2023-11-17T00:08:43.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 -- start start 2023-11-17T00:08:43.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 Processor -- start 2023-11-17T00:08:43.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08ca9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca1330 0x559a08cb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08b92960 con 0x559a08c9ff10 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08b92600 con 0x559a08ca1330 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08a749a0 con 0x559a08ca0860 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.632+0000 7fda465fdb80 10 client.0 _ll_register_callbacks cb 0x559a08bb0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08ca9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3099b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3099b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49432/0 (socket says 172.21.15.6:49432) 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3099b700 1 -- 172.21.15.6:0/664793582 learned_addr learned my addr 172.21.15.6:0/664793582 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca1330 0x559a08cb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 -- 172.21.15.6:0/664793582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 msgr2=0x559a08ca9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 -- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca1330 msgr2=0x559a08cb0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca1330 0x559a08cb0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 -- 172.21.15.6:0/664793582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda20009cd0 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda3119c700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08ca9820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda24009fd0 tx=0x7fda2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e38286178e5f155 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.633+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400e070 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda24007be0 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a08c92df0 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda24007da0 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 10 client.15381 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a08bb7210 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a08c92df0 con 0x559a08c9ff10 2023-11-17T00:08:43.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda24004630 con 0x559a08c9ff10 2023-11-17T00:08:43.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 10 client.15381 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:43.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:08:43.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 20 client.15381 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:43.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.634+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08cb5590 0x559a08cb79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.635+0000 7fda465fdb80 4 client.15381 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:08:43.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.635+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a08bb3120 con 0x559a08cb5590 2023-11-17T00:08:43.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.635+0000 7fda3099b700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08cb5590 0x559a08cb79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.635+0000 7fda3099b700 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08cb5590 0x559a08cb79a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a2c3a20d2bcaf7 server_cookie=72c3276989961ef5 in_seq=0 out_seq=0 2023-11-17T00:08:43.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.635+0000 7fda0bfff700 10 client.15381 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.636+0000 7fda0bfff700 1 -- 172.21.15.6:0/664793582 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a08bb3120 con 0x559a08cb5590 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.636+0000 7fda0bfff700 10 client.15381 handle_command_reply: tid=0 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.636+0000 7fda465fdb80 1 client.15381 shutdown 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.636+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 msgr2=0x559a08ca9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.636+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08ca9820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda24009fd0 tx=0x7fda2400b040 comp rx=0 tx=0).stop 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 shutdown_connections 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08cb5590 0x559a08cb79a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08ca9820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca1330 0x559a08cb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 --2- 172.21.15.6:0/664793582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08ca0860 0x559a08ca9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 >> 172.21.15.6:0/664793582 conn(0x559a08c95630 msgr2=0x559a08c9a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda0bfff700 0 client.15381 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 shutdown_connections 2023-11-17T00:08:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 1 -- 172.21.15.6:0/664793582 wait complete. 2023-11-17T00:08:43.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.637+0000 7fda465fdb80 20 client.15381 trim_cache size 0 max 16384 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 Processor -- start 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 -- start start 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 0x559a08ca7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca0860 0x559a08ca7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08cb2a90 con 0x559a08c9ff10 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08caa860 con 0x559a08ca0860 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda465fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08ca8cd0 con 0x559a08c95630 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda3099b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda3099b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60658/0 (socket says 172.21.15.6:60658) 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda3099b700 1 -- 172.21.15.6:0/1485673073 learned_addr learned my addr 172.21.15.6:0/1485673073 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.655+0000 7fda3119c700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 0x559a08ca7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3119c700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca0860 0x559a08ca7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 -- 172.21.15.6:0/1485673073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 msgr2=0x559a08ca7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 0x559a08ca7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 -- 172.21.15.6:0/1485673073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca0860 msgr2=0x559a08ca7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca0860 0x559a08ca7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 -- 172.21.15.6:0/1485673073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a08ca19d0 con 0x559a08c9ff10 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3119c700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 0x559a08ca7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda3099b700 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda20005950 tx=0x7fda20009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4d178bd559390b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda177fe700 1 -- 172.21.15.6:0/1485673073 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2001e040 con 0x559a08c9ff10 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda177fe700 1 -- 172.21.15.6:0/1485673073 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2001f040 con 0x559a08c9ff10 2023-11-17T00:08:43.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.656+0000 7fda177fe700 1 -- 172.21.15.6:0/1485673073 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20003d40 con 0x559a08c9ff10 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 -- 172.21.15.6:0/1485673073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 msgr2=0x559a08cb0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda20005950 tx=0x7fda20009fa0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 -- 172.21.15.6:0/1485673073 shutdown_connections 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08ca0860 0x559a08ca7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c95630 0x559a08ca7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 --2- 172.21.15.6:0/1485673073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08c9ff10 0x559a08cb0d60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 -- 172.21.15.6:0/1485673073 >> 172.21.15.6:0/1485673073 conn(0x559a08ca88d0 msgr2=0x559a08cbc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 -- 172.21.15.6:0/1485673073 shutdown_connections 2023-11-17T00:08:43.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.657+0000 7fda465fdb80 1 -- 172.21.15.6:0/1485673073 wait complete. 2023-11-17T00:08:43.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 -- start start 2023-11-17T00:08:43.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 Processor -- start 2023-11-17T00:08:43.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c95630 0x559a08cada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c9ff10 0x559a08cadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08cae4e0 con 0x559a08ca0860 2023-11-17T00:08:43.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08cae650 con 0x559a08c95630 2023-11-17T00:08:43.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a08d09980 con 0x559a08c9ff10 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda465fdb80 10 client.0 _ll_register_callbacks cb 0x7fda24008200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.658+0000 7fda3119c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60662/0 (socket says 172.21.15.6:60662) 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 -- 172.21.15.6:0/4164869315 learned_addr learned my addr 172.21.15.6:0/4164869315 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c95630 0x559a08cada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3099b700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c9ff10 0x559a08cadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 -- 172.21.15.6:0/4164869315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c9ff10 msgr2=0x559a08cadfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c9ff10 0x559a08cadfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 -- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c95630 msgr2=0x559a08cada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c95630 0x559a08cada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 -- 172.21.15.6:0/4164869315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a08a726b0 con 0x559a08ca0860 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda3119c700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda24008950 tx=0x7fda2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da9b48a4a7265cc server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400e040 con 0x559a08ca0860 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda200044f0 con 0x559a08ca0860 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.659+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda24007df0 con 0x559a08ca0860 2023-11-17T00:08:43.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda24004bb0 con 0x559a08ca0860 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 10 client.15387 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a08a1ee60 con 0x559a08ca0860 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda200044f0 con 0x559a08ca0860 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda2401a6d0 con 0x559a08ca0860 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 10 client.15387 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 20 client.15387 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08ca1330 0x559a08d0d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 4 client.15387 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559a08bbf6e0 con 0x559a08ca1330 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.660+0000 7fda3099b700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08ca1330 0x559a08d0d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.661+0000 7fda3099b700 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08ca1330 0x559a08d0d2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc01636f0e4c5fb7 server_cookie=2ad4061760687886 in_seq=0 out_seq=0 2023-11-17T00:08:43.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.661+0000 7fda0bfff700 10 client.15387 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda0bfff700 1 -- 172.21.15.6:0/4164869315 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559a08bbf6e0 con 0x559a08ca1330 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda0bfff700 10 client.15387 handle_command_reply: tid=0 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 client.15387 shutdown 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 msgr2=0x559a08d09440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda24008950 tx=0x7fda2400b040 comp rx=0 tx=0).stop 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 shutdown_connections 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559a08ca1330 0x559a08d0d2d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559a08ca0860 0x559a08d09440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559a08c95630 0x559a08cada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 --2- 172.21.15.6:0/4164869315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559a08c9ff10 0x559a08cadfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.662+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 >> 172.21.15.6:0/4164869315 conn(0x559a08ca88d0 msgr2=0x559a08ca4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda0bfff700 0 client.15387 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 shutdown_connections 2023-11-17T00:08:43.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda465fdb80 1 -- 172.21.15.6:0/4164869315 wait complete. 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda465fdb80 20 client.15387 trim_cache size 0 max 16384 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "9c81b619-dd1f-4e79-b9b3-bb6d98e6e835", 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:08:43.670 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 msgr2=0x7fda1c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 0x7fda1c0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda180058b0 tx=0x7fda18009c00 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 msgr2=0x7fda2c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.663+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fda18009fd0 tx=0x7fda1800b790 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 shutdown_connections 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2c101be0 0x7fda2c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda2c100060 0x7fda2c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 --2- 172.21.15.6:0/3476204805 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda2c101250 0x7fda2c198520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 >> 172.21.15.6:0/3476204805 conn(0x7fda2c0fb6a0 msgr2=0x7fda2c106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 shutdown_connections 2023-11-17T00:08:43.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.664+0000 7fda3199d700 1 -- 172.21.15.6:0/3476204805 wait complete. 2023-11-17T00:08:43.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:43 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2151350825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:43 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2569964673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.683 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:08:43.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:43 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2151350825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:43 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2569964673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:43 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2151350825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:43 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2569964673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/2558479925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4105db0 msgr2=0x7fd2f4108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/2558479925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4105db0 0x7fd2f4108190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b0a0 tx=0x7fd2ec00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/2558479925 shutdown_connections 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/2558479925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4105db0 0x7fd2f4108190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/2558479925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4100b70 0x7fd2f4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/2558479925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4103490 0x7fd2f4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/2558479925 >> 172.21.15.6:0/2558479925 conn(0x7fd2f40fa570 msgr2=0x7fd2f40fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/2558479925 shutdown_connections 2023-11-17T00:08:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.890+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/2558479925 wait complete. 2023-11-17T00:08:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.891+0000 7fd2f8d9d700 1 Processor -- start 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.891+0000 7fd2f8d9d700 1 -- start start 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4100b70 0x7fd2f4195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 0x7fd2f419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419da80 con 0x7fd2f4103490 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419dbc0 con 0x7fd2f4100b70 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f8d9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419dd50 con 0x7fd2f4105db0 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60682/0 (socket says 172.21.15.6:60682) 2023-11-17T00:08:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/902244809 learned_addr learned my addr 172.21.15.6:0/902244809 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4100b70 0x7fd2f4195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 0x7fd2f419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/902244809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 msgr2=0x7fd2f419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 0x7fd2f419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/902244809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4100b70 msgr2=0x7fd2f4195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4100b70 0x7fd2f4195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/902244809 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009d00 con 0x7fd2f4103490 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.892+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 0x7fd2f419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e9a0 tx=0x7fd2e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f0ff9700 1 -- 172.21.15.6:0/902244809 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8009790 con 0x7fd2f4103490 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f0ff9700 1 -- 172.21.15.6:0/902244809 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e800cbd0 con 0x7fd2f4103490 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f0ff9700 1 -- 172.21.15.6:0/902244809 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8018680 con 0x7fd2f4103490 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f410ba30 con 0x7fd2f4103490 2023-11-17T00:08:43.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.893+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f410b8a0 con 0x7fd2f4103490 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.896+0000 7fd2f0ff9700 1 -- 172.21.15.6:0/902244809 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd2e800cd40 con 0x7fd2f4103490 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.898+0000 7fd2f0ff9700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 0x7fd2dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.898+0000 7fd2f0ff9700 1 -- 172.21.15.6:0/902244809 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2e8014070 con 0x7fd2f4103490 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.898+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 0x7fd2dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.899+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 0x7fd2dc0a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4068880 tx=0x7fd2e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.900+0000 7fd30d9e1b80 1 Processor -- start 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.900+0000 7fd30d9e1b80 1 -- start start 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f512b0 0x55ce90f51750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90e55060 con 0x55ce90f4fe90 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90ea78a0 con 0x55ce90f507e0 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90ea0d30 con 0x55ce90f512b0 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60686/0 (socket says 172.21.15.6:60686) 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 -- 172.21.15.6:0/3256155649 learned_addr learned my addr 172.21.15.6:0/3256155649 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f512b0 0x55ce90f51750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 -- 172.21.15.6:0/3256155649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f512b0 msgr2=0x55ce90f51750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f512b0 0x55ce90f51750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 -- 172.21.15.6:0/3256155649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 msgr2=0x55ce90f50d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f50d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.901+0000 7fd2f37fe700 1 -- 172.21.15.6:0/3256155649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce90d22630 con 0x55ce90f4fe90 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.902+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.902+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4011d50 tx=0x7fd2e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7488b4f1a41da216 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.902+0000 7fd2d9ffb700 1 -- 172.21.15.6:0/3256155649 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e400eb20 con 0x55ce90f4fe90 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.902+0000 7fd2d9ffb700 1 -- 172.21.15.6:0/3256155649 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e4021070 con 0x55ce90f4fe90 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.902+0000 7fd2d9ffb700 1 -- 172.21.15.6:0/3256155649 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e401d670 con 0x55ce90f4fe90 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3256155649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 msgr2=0x55ce90f502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4011d50 tx=0x7fd2e4014000 comp rx=0 tx=0).stop 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3256155649 shutdown_connections 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f4fe90 0x55ce90f502a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f512b0 0x55ce90f51750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3256155649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f50d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3256155649 >> 172.21.15.6:0/3256155649 conn(0x55ce90f455b0 msgr2=0x55ce90f4a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3256155649 shutdown_connections 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.903+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3256155649 wait complete. 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 -- start start 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 Processor -- start 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 0x55ce90f5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90e428e0 con 0x55ce90f512b0 2023-11-17T00:08:43.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90e42580 con 0x55ce90f507e0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90d24920 con 0x55ce90f4fe90 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.904+0000 7fd30d9e1b80 10 client.0 _ll_register_callbacks cb 0x55ce90e60780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 0x55ce90f5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33142/0 (socket says 172.21.15.6:33142) 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/2603949895 learned_addr learned my addr 172.21.15.6:0/2603949895 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/2603949895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 msgr2=0x55ce90f5dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 0x55ce90f5dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/2603949895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 msgr2=0x55ce90f65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 -- 172.21.15.6:0/2603949895 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e4017070 con 0x55ce90f507e0 2023-11-17T00:08:43.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 0x55ce90f5dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8096440 tx=0x7fd2e809a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7c771240eaacd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.905+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80a2100 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce90f42d70 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2e80a2390 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e80b80d0 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 10 client.25448 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ce90e67190 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce90f42d70 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2e8095d30 con 0x55ce90f507e0 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:43.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 20 client.25448 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90f69910 0x55ce90f6bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 4 client.25448 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.906+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce90e630a0 con 0x55ce90f69910 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.907+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90f69910 0x55ce90f6bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.907+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90f69910 0x55ce90f6bd20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c214bc96b69cc51 server_cookie=2db19a4c174c3023 in_seq=0 out_seq=0 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.907+0000 7fd2d2ffd700 10 client.25448 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2603949895 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce90e630a0 con 0x55ce90f69910 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd2d2ffd700 10 client.25448 handle_command_reply: tid=0 2023-11-17T00:08:43.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 client.25448 shutdown 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 msgr2=0x55ce90f5e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8096440 tx=0x7fd2e809a480 comp rx=0 tx=0).stop 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 shutdown_connections 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90f69910 0x55ce90f6bd20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f4fe90 0x55ce90f5dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2603949895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f507e0 0x55ce90f5e0f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 >> 172.21.15.6:0/2603949895 conn(0x55ce90f455b0 msgr2=0x55ce90f4a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd2d2ffd700 0 client.25448 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 shutdown_connections 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.908+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2603949895 wait complete. 2023-11-17T00:08:43.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.909+0000 7fd30d9e1b80 20 client.25448 trim_cache size 0 max 16384 2023-11-17T00:08:43.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.910+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f4064061700 con 0x7f4064074a90 2023-11-17T00:08:43.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.926+0000 7fd30d9e1b80 1 Processor -- start 2023-11-17T00:08:43.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.926+0000 7fd30d9e1b80 1 -- start start 2023-11-17T00:08:43.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f5d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f4c2e0 con 0x55ce90f512b0 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f45370 con 0x55ce90f4fe90 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f5ed30 con 0x55ce90f507e0 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60714/0 (socket says 172.21.15.6:60714) 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd2f3fff700 1 -- 172.21.15.6:0/3940943291 learned_addr learned my addr 172.21.15.6:0/3940943291 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f5d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.927+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 -- 172.21.15.6:0/3940943291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 msgr2=0x55ce90f650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 -- 172.21.15.6:0/3940943291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 msgr2=0x55ce90f5d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f5d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:43.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 -- 172.21.15.6:0/3940943291 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce90e66ee0 con 0x55ce90f512b0 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f5d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b0a0 tx=0x7fd2ec007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcefdaa5da45789 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2da7fc700 1 -- 172.21.15.6:0/3940943291 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00e040 con 0x55ce90f512b0 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2da7fc700 1 -- 172.21.15.6:0/3940943291 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec0092e0 con 0x55ce90f512b0 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.928+0000 7fd2da7fc700 1 -- 172.21.15.6:0/3940943291 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec005040 con 0x55ce90f512b0 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3940943291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 msgr2=0x55ce90f5dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b0a0 tx=0x7fd2ec007d10 comp rx=0 tx=0).stop 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3940943291 shutdown_connections 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f5dea0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f5d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/3940943291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3940943291 >> 172.21.15.6:0/3940943291 conn(0x55ce90f455b0 msgr2=0x55ce90f70680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3940943291 shutdown_connections 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/3940943291 wait complete. 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.929+0000 7fd30d9e1b80 1 -- start start 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 Processor -- start 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f64840 con 0x55ce90f512b0 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f649b0 con 0x55ce90f4fe90 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce90f64b20 con 0x55ce90f507e0 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.930+0000 7fd30d9e1b80 10 client.0 _ll_register_callbacks cb 0x7fd2e80998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60726/0 (socket says 172.21.15.6:60726) 2023-11-17T00:08:44.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f3fff700 1 -- 172.21.15.6:0/2273593422 learned_addr learned my addr 172.21.15.6:0/2273593422 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 -- 172.21.15.6:0/2273593422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 msgr2=0x55ce90f617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 -- 172.21.15.6:0/2273593422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 msgr2=0x55ce90f64300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 -- 172.21.15.6:0/2273593422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e402a040 con 0x55ce90f4fe90 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f3fff700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2f37fe700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f61280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2e400e970 tx=0x7fd2e400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e8c9d0c89419d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.931+0000 7fd2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e400ea60 con 0x55ce90f4fe90 2023-11-17T00:08:44.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2e4021070 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2e401cac0 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e401cca0 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd30d9e1b80 10 client.25454 fetch_fsmap learned FSMap version 20 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ce90e67190 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.932+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e4021070 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2e40320f0 con 0x55ce90f4fe90 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 10 client.25454 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 20 client.25454 populate_metadata read hostname 'smithi006' 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90fbe6b0 0x55ce90fc0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 4 client.25454 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce90e6f660 con 0x55ce90fbe6b0 2023-11-17T00:08:44.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90fbe6b0 0x55ce90fc0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd2f2ffd700 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90fbe6b0 0x55ce90fc0ac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214af47efac80d83 server_cookie=5ec94c9ae864dda1 in_seq=0 out_seq=0 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.933+0000 7fd2d2ffd700 10 client.25454 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd2d2ffd700 1 -- 172.21.15.6:0/2273593422 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ce90e6f660 con 0x55ce90fbe6b0 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd2d2ffd700 10 client.25454 handle_command_reply: tid=0 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 client.25454 shutdown 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 msgr2=0x55ce90f61280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f61280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2e400e970 tx=0x7fd2e400ee90 comp rx=0 tx=0).stop 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 shutdown_connections 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ce90fbe6b0 0x55ce90fc0ac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce90f512b0 0x55ce90f64300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ce90f4fe90 0x55ce90f61280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 --2- 172.21.15.6:0/2273593422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce90f507e0 0x55ce90f617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd2d2ffd700 0 client.25454 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 >> 172.21.15.6:0/2273593422 conn(0x55ce90f455b0 msgr2=0x55ce90f5d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 shutdown_connections 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 1 -- 172.21.15.6:0/2273593422 wait complete. 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.934+0000 7fd30d9e1b80 20 client.25454 trim_cache size 0 max 16384 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stdout: "9c81b619-dd1f-4e79-b9b3-bb6d98e6e835": { 2023-11-17T00:08:44.030 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:08:44.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9c81b619-dd1f-4e79-b9b3-bb6d98e6e835", 2023-11-17T00:08:44.031 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:08:44.031 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:08:44.031 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:08:44.031 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 msgr2=0x7fd2dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 0x7fd2dc0a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4068880 tx=0x7fd2e4009260 comp rx=0 tx=0).stop 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 msgr2=0x7fd2f4196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e9a0 tx=0x7fd2e800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 shutdown_connections 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7fd2dc0a11d0 0x7fd2dc0a3680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f4105db0 0x7fd2f419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f4100b70 0x7fd2f4195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 --2- 172.21.15.6:0/902244809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f4103490 0x7fd2f4196440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 >> 172.21.15.6:0/902244809 conn(0x7fd2f40fa570 msgr2=0x7fd2f41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 shutdown_connections 2023-11-17T00:08:44.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:43.935+0000 7fd2f8d9d700 1 -- 172.21.15.6:0/902244809 wait complete. 2023-11-17T00:08:44.033 INFO:tasks.cephfs.filesystem:scrub status for tag:9c81b619-dd1f-4e79-b9b3-bb6d98e6e835 - {'path': '/', 'tag': '9c81b619-dd1f-4e79-b9b3-bb6d98e6e835', 'options': 'recursive,force'} 2023-11-17T00:08:44.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.515+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f4054096630 con 0x7f4064074a90 2023-11-17T00:08:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.517+0000 7f405b7fe700 1 -- 172.21.15.6:0/2852292195 <== mon.2 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f4054067760 con 0x7f4064074a90 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 msgr2=0x7f40500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 0x7f40500a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f406000b2e0 tx=0x7f4060005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 msgr2=0x7f406419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 0x7f406419cb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f405400e9f0 tx=0x7f405400ed00 comp rx=0 tx=0).stop 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 shutdown_connections 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f40500a11d0 0x7f40500a3680 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40640734c0 0x7f406419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4064103ca0 0x7f40641a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 --2- 172.21.15.6:0/2852292195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4064074a90 0x7f406419cb30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 >> 172.21.15.6:0/2852292195 conn(0x7f40640fb810 msgr2=0x7f40640fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 shutdown_connections 2023-11-17T00:08:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.520+0000 7f406c3ff700 1 -- 172.21.15.6:0/2852292195 wait complete. 2023-11-17T00:08:44.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-11-17T00:08:44.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2910868019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28101dd0 msgr2=0x7f4d28108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2910868019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28101dd0 0x7f4d28108690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009f90 tx=0x7f4d2000b330 comp rx=0 tx=0).stop 2023-11-17T00:08:44.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2910868019 shutdown_connections 2023-11-17T00:08:44.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2910868019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28101dd0 0x7f4d28108690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2910868019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28100250 0x7f4d28100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2910868019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101440 0x7f4d28101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2910868019 >> 172.21.15.6:0/2910868019 conn(0x7f4d280fb850 msgr2=0x7f4d280fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2910868019 shutdown_connections 2023-11-17T00:08:44.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.760+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2910868019 wait complete. 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.762+0000 7f4d2cfff700 1 Processor -- start 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.762+0000 7f4d2cfff700 1 -- start start 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28101440 0x7f4d28196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 0x7f4d2819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819dc70 con 0x7f4d28101dd0 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819ddb0 con 0x7f4d28100250 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819df40 con 0x7f4d28101440 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33182/0 (socket says 172.21.15.6:33182) 2023-11-17T00:08:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 -- 172.21.15.6:0/2259269964 learned_addr learned my addr 172.21.15.6:0/2259269964 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d25d9b700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28101440 0x7f4d28196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d26d9d700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 0x7f4d2819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28101440 msgr2=0x7f4d28196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28101440 0x7f4d28196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 msgr2=0x7f4d2819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 0x7f4d2819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d2659c700 1 -- 172.21.15.6:0/2259269964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d20009c40 con 0x7f4d28100250 2023-11-17T00:08:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.763+0000 7f4d26d9d700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 0x7f4d2819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: ignoring --setuser ceph since I am not root 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: ignoring --setgroup ceph since I am not root 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mgr[132666]: -- 172.21.15.87:0/4010376824 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bb2f15d520 con 0x55bb2f133000 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:44.683+0000 7fdcda6b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:44.763+0000 7fdcda6b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:08:44.766 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:08:44.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:44.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2852292195' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.767 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d2659c700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e970 tx=0x7f4d1800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:44.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18009790 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d1800eeb0 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2810bc30 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003d20 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.764+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2810baa0 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.765+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d281077a0 con 0x7f4d28100250 2023-11-17T00:08:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.767+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f4d18004870 con 0x7f4d28100250 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2852292195' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.771 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.772 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: ignoring --setuser ceph since I am not root 2023-11-17T00:08:44.772 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: ignoring --setgroup ceph since I am not root 2023-11-17T00:08:44.772 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-mgr[133659]: -- 172.21.15.186:0/4158292754 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e991b9520 con 0x558e9918e800 2023-11-17T00:08:44.772 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:44.687+0000 7f5a4d624200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-17T00:08:44.772 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:44.766+0000 7f5a4d624200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.768+0000 7f4d177fe700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 0x7f4d100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.768+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4d18014070 con 0x7f4d28100250 2023-11-17T00:08:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.768+0000 7f4d25d9b700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 msgr2=0x7f4d100a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.768+0000 7f4d25d9b700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 0x7f4d100a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-17T00:08:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.770+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4d180968a0 con 0x7f4d28100250 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 3 op/s 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2852292195' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.799 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-17T00:08:44.800 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setuser ceph since I am not root 2023-11-17T00:08:44.800 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: ignoring --setgroup ceph since I am not root 2023-11-17T00:08:44.800 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-mgr[98110]: -- 172.21.15.6:0/205463707 <== mon.2 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55744d0f9520 con 0x55744d0d0c00 2023-11-17T00:08:44.800 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:44.798+0000 7f9c745d4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-17T00:08:44.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.969+0000 7f4d25d9b700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 msgr2=0x7f4d100a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:44.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.969+0000 7f4d25d9b700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 0x7f4d100a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-17T00:08:44.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.984+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4d28061980 con 0x7f4d28100250 2023-11-17T00:08:44.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.989+0000 7f4d177fe700 1 -- 172.21.15.6:0/2259269964 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4d18096020 con 0x7f4d28100250 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 msgr2=0x7f4d100a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 0x7f4d100a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 msgr2=0x7f4d281960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e970 tx=0x7f4d1800ed30 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 shutdown_connections 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f4d100a1220 0x7f4d100a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d28101dd0 0x7f4d2819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d28100250 0x7f4d281960f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 --2- 172.21.15.6:0/2259269964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d28101440 0x7f4d28196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 >> 172.21.15.6:0/2259269964 conn(0x7f4d280fb850 msgr2=0x7f4d281048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 shutdown_connections 2023-11-17T00:08:44.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:44.992+0000 7f4d2cfff700 1 -- 172.21.15.6:0/2259269964 wait complete. 2023-11-17T00:08:45.012 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-11-17T00:08:45.082 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:44.908+0000 7f9c745d4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:45.082 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:45.030+0000 7f9c745d4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-17T00:08:45.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:44.854+0000 7fdcda6b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-17T00:08:45.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:44.946+0000 7fdcda6b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-17T00:08:45.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:44.858+0000 7f5a4d624200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-17T00:08:45.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:44 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:44.951+0000 7f5a4d624200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-17T00:08:45.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 -- 172.21.15.6:0/2177475011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5178100cc0 msgr2=0x7f51781030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 --2- 172.21.15.6:0/2177475011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5178100cc0 0x7f51781030a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 -- 172.21.15.6:0/2177475011 shutdown_connections 2023-11-17T00:08:45.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 --2- 172.21.15.6:0/2177475011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f5178108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 --2- 172.21.15.6:0/2177475011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5178100cc0 0x7f51781030a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 --2- 172.21.15.6:0/2177475011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51781035e0 0x7f51781059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 -- 172.21.15.6:0/2177475011 >> 172.21.15.6:0/2177475011 conn(0x7f51780fa720 msgr2=0x7f51780fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 -- 172.21.15.6:0/2177475011 shutdown_connections 2023-11-17T00:08:45.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.224+0000 7f517d9ff700 1 -- 172.21.15.6:0/2177475011 wait complete. 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.225+0000 7f517d9ff700 1 Processor -- start 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.225+0000 7f517d9ff700 1 -- start start 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178100cc0 0x7f51781981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 0x7f51781986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f517819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819fd20 con 0x7f5178100cc0 2023-11-17T00:08:45.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819fe60 con 0x7f51781035e0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f517d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819fff0 con 0x7f5178105f00 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f51777fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f517819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178100cc0 0x7f51781981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f51777fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f517819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49546/0 (socket says 172.21.15.6:49546) 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f51777fe700 1 -- 172.21.15.6:0/1172957878 learned_addr learned my addr 172.21.15.6:0/1172957878 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f51767fc700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 0x7f51781986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.226+0000 7f51767fc700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 msgr2=0x7f517819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f51767fc700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f517819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f51767fc700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178100cc0 msgr2=0x7f51781981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f51767fc700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178100cc0 0x7f51781981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f51767fc700 1 -- 172.21.15.6:0/1172957878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680095f0 con 0x7f51781035e0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f51767fc700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 0x7f51781986e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:45.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c00c9c0 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516c010430 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c018680 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517810bbf0 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.227+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517810ba60 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.229+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178107af0 con 0x7f51781035e0 2023-11-17T00:08:45.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.230+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f516c010860 con 0x7f51781035e0 2023-11-17T00:08:45.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.231+0000 7f517c9fd700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 0x7f51600a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.232+0000 7f5176ffd700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 msgr2=0x7f51600a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:45.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.232+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f516c014070 con 0x7f51781035e0 2023-11-17T00:08:45.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.232+0000 7f5176ffd700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 0x7f51600a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-17T00:08:45.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.235+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f516c051730 con 0x7f51781035e0 2023-11-17T00:08:45.397 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:45.162+0000 7f9c745d4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-17T00:08:45.423 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:45.117+0000 7fdcda6b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-17T00:08:45.431 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:45.121+0000 7f5a4d624200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-17T00:08:45.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.433+0000 7f5176ffd700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 msgr2=0x7f51600a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:45.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.433+0000 7f5176ffd700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 0x7f51600a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-17T00:08:45.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.443+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f517802cfe0 con 0x7f51781035e0 2023-11-17T00:08:45.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.449+0000 7f517c9fd700 1 -- 172.21.15.6:0/1172957878 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f516c0961e0 con 0x7f51781035e0 2023-11-17T00:08:45.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.451+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 msgr2=0x7f51600a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.451+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 0x7f51600a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 msgr2=0x7f51781986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 0x7f51781986e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00ed00 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 shutdown_connections 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f51600a11a0 0x7f51600a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5178100cc0 0x7f51781981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178105f00 0x7f517819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 --2- 172.21.15.6:0/1172957878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51781035e0 0x7f51781986e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 >> 172.21.15.6:0/1172957878 conn(0x7f51780fa720 msgr2=0x7f51780ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 shutdown_connections 2023-11-17T00:08:45.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.452+0000 7f517d9ff700 1 -- 172.21.15.6:0/1172957878 wait complete. 2023-11-17T00:08:45.477 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 -- 172.21.15.6:0/1423590273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6878103ca0 msgr2=0x7f687810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 --2- 172.21.15.6:0/1423590273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6878103ca0 0x7f687810e8d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6870009f90 tx=0x7f687000b330 comp rx=0 tx=0).stop 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 -- 172.21.15.6:0/1423590273 shutdown_connections 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 --2- 172.21.15.6:0/1423590273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6878103ca0 0x7f687810e8d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 --2- 172.21.15.6:0/1423590273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878074a90 0x7f6878072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 --2- 172.21.15.6:0/1423590273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68780734c0 0x7f6878073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 -- 172.21.15.6:0/1423590273 >> 172.21.15.6:0/1423590273 conn(0x7f68780fb810 msgr2=0x7f68780fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:45.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 -- 172.21.15.6:0/1423590273 shutdown_connections 2023-11-17T00:08:45.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.685+0000 7f687fdff700 1 -- 172.21.15.6:0/1423590273 wait complete. 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 Processor -- start 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 -- start start 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68780734c0 0x7f687819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 0x7f687819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878103ca0 0x7f68781a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.686+0000 7f687fdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781a4260 con 0x7f68780734c0 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687fdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781a43a0 con 0x7f6878103ca0 2023-11-17T00:08:45.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781a4530 con 0x7f6878074a90 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 0x7f687819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878103ca0 0x7f68781a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68780734c0 0x7f687819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 0x7f687819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:49572/0 (socket says 172.21.15.6:49572) 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687d39a700 1 -- 172.21.15.6:0/2846287289 learned_addr learned my addr 172.21.15.6:0/2846287289 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 msgr2=0x7f687819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 0x7f687819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68780734c0 msgr2=0x7f687819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68780734c0 0x7f687819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:45.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687e39c700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68680095f0 con 0x7f6878103ca0 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.687+0000 7f687d39a700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6878074a90 0x7f687819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f687e39c700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878103ca0 0x7f68781a3c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f687000b6c0 tx=0x7f6870004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000e040 con 0x7f6878103ca0 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f687000bbf0 con 0x7f6878103ca0 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6870009c40 con 0x7f6878103ca0 2023-11-17T00:08:45.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870005040 con 0x7f6878103ca0 2023-11-17T00:08:45.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.688+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68781121f0 con 0x7f6878103ca0 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.689+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6878112380 con 0x7f6878103ca0 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.691+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f687001d470 con 0x7f6878103ca0 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.692+0000 7f686effd700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.692+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f68700ce4e0 con 0x7f6878103ca0 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.692+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.692+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-17T00:08:45.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.694+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6870097a10 con 0x7f6878103ca0 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-mon[97840]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:45.395+0000 7f9c745d4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-17T00:08:45.738 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-17T00:08:45.738 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-mon[131182]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-11-17T00:08:45.738 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.738 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:08:45.738 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.739 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:45.421+0000 7fdcda6b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-17T00:08:45.739 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:45.512+0000 7fdcda6b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-17T00:08:45.739 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:45.640+0000 7fdcda6b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-17T00:08:45.739 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:45.735+0000 7fdcda6b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-17T00:08:45.748 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:45.428+0000 7f5a4d624200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-17T00:08:45.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:45.519+0000 7f5a4d624200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-17T00:08:45.749 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:45.649+0000 7f5a4d624200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-17T00:08:45.750 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-17T00:08:45.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-mon[131487]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-11-17T00:08:45.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.751 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:08:45.752 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' 2023-11-17T00:08:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.893+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.893+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-17T00:08:45.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:45.931+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6878060f30 con 0x7f68640a11a0 2023-11-17T00:08:46.099 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:45.809+0000 7f9c745d4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-17T00:08:46.099 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:45.929+0000 7f9c745d4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-17T00:08:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:46.294+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:46.294+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-17T00:08:46.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:46.098+0000 7f9c745d4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-17T00:08:46.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:46.227+0000 7f9c745d4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-17T00:08:46.601 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:46.155+0000 7fdcda6b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-17T00:08:46.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:45.744+0000 7f5a4d624200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-17T00:08:46.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:46.168+0000 7f5a4d624200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-17T00:08:47.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:47.095+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:47.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:47.095+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-17T00:08:47.100 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:46.716+0000 7fdcda6b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-17T00:08:47.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:46.883+0000 7fdcda6b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-17T00:08:47.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:46.730+0000 7f5a4d624200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-17T00:08:47.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:46.901+0000 7f5a4d624200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-17T00:08:47.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:46.784+0000 7f9c745d4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-17T00:08:47.899 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:47.639+0000 7fdcda6b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:47.900 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:47.729+0000 7fdcda6b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-17T00:08:47.929 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:47.666+0000 7f5a4d624200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:47.930 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:47.757+0000 7f5a4d624200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-17T00:08:47.972 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:47.558+0000 7f9c745d4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-17T00:08:47.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:47.773+0000 7f9c745d4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-17T00:08:48.173 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:47.898+0000 7fdcda6b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:48.205 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:47.928+0000 7f5a4d624200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:48.543 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:48.172+0000 7fdcda6b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-17T00:08:48.543 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:48.395+0000 7fdcda6b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-17T00:08:48.543 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:48.541+0000 7fdcda6b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-17T00:08:48.580 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:48.204+0000 7f5a4d624200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-17T00:08:48.580 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:48.431+0000 7f5a4d624200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-17T00:08:48.580 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:48.578+0000 7f5a4d624200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-17T00:08:48.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:48.697+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:48.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:48.697+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-17T00:08:49.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:48.801+0000 7fdcda6b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-17T00:08:49.115 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:48.839+0000 7f5a4d624200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-17T00:08:49.121 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:48.759+0000 7f9c745d4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:49.121 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:48.885+0000 7f9c745d4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-17T00:08:49.472 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:49.120+0000 7f9c745d4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-17T00:08:49.785 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:49.498+0000 7f9c745d4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-17T00:08:49.818 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:49.499+0000 7fdcda6b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-17T00:08:49.818 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:49.629+0000 7fdcda6b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-17T00:08:49.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:49.548+0000 7f5a4d624200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-17T00:08:49.865 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:49.679+0000 7f5a4d624200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-17T00:08:50.100 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:49.817+0000 7fdcda6b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-17T00:08:50.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:49.930+0000 7fdcda6b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:50.200 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:49.869+0000 7f5a4d624200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-17T00:08:50.200 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:49.982+0000 7f5a4d624200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:50.222 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:49.784+0000 7f9c745d4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-17T00:08:50.223 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:49.986+0000 7f9c745d4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-17T00:08:50.601 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:50.143+0000 7fdcda6b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-17T00:08:50.615 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:50.198+0000 7f5a4d624200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-17T00:08:50.722 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:50.345+0000 7f9c745d4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-17T00:08:51.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:51.065+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 20) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f6870097de0 con 0x7f6878103ca0 2023-11-17T00:08:51.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: Standby manager daemon y restarted 2023-11-17T00:08:51.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: Standby manager daemon y started 2023-11-17T00:08:51.323 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:08:51.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:51.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:08:51.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:51.351 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:08:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y[132642]: 2023-11-17T00:08:50.989+0000 7fdcda6b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-17T00:08:51.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: Standby manager daemon y restarted 2023-11-17T00:08:51.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: Standby manager daemon y started 2023-11-17T00:08:51.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:08:51.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:51.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:08:51.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:51 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:51.365 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z[133635]: 2023-11-17T00:08:51.074+0000 7f5a4d624200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: Standby manager daemon y restarted 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: Standby manager daemon y started 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-17T00:08:51.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:51 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.87:0/284174169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:51.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:51.322+0000 7f9c745d4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-17T00:08:51.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:51.503+0000 7f9c745d4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-17T00:08:51.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:51.898+0000 7f687db9b700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/879660796 2023-11-17T00:08:51.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:51.898+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-17T00:08:52.069 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:51.762+0000 7f9c745d4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-17T00:08:52.069 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:51.918+0000 7f9c745d4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-17T00:08:52.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:52.082+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 21) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f68700cbc90 con 0x7f6878103ca0 2023-11-17T00:08:52.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-11-17T00:08:52.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: Standby manager daemon z restarted 2023-11-17T00:08:52.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: Standby manager daemon z started 2023-11-17T00:08:52.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:08:52.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:52.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:08:52.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:52 smithi087 ceph-mon[131182]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:52.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-11-17T00:08:52.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: Standby manager daemon z restarted 2023-11-17T00:08:52.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: Standby manager daemon z started 2023-11-17T00:08:52.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:08:52.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:52.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:08:52.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:52 smithi186 ceph-mon[131487]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:52.214+0000 7f9c745d4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: Standby manager daemon z restarted 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: Standby manager daemon z started 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-17T00:08:52.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:52 smithi006 ceph-mon[97840]: from='mgr.? 172.21.15.186:0/1837781081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-17T00:08:53.333 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-mon[97840]: mgrmap e21: x(active, since 9m), standbys: z, y 2023-11-17T00:08:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:53 smithi087 ceph-mon[131182]: mgrmap e21: x(active, since 9m), standbys: z, y 2023-11-17T00:08:53.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:53.352+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 9 ==== mgrmap(e 22) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f687009abc0 con 0x7f6878103ca0 2023-11-17T00:08:53.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:53.352+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 msgr2=0x7f68640a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:08:53.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:53.352+0000 7f686effd700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/879660796,v1:172.21.15.6:6801/879660796] conn(0x7f68640a11a0 0x7f68640a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:53 smithi186 ceph-mon[131487]: mgrmap e21: x(active, since 9m), standbys: z, y 2023-11-17T00:08:53.586 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.331+0000 7f9c745d4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.584+0000 7f9bcf269700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.584+0000 7f9bcf269700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.584+0000 7f9bcf269700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.584+0000 7f9bcf269700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.584+0000 7f9bcf269700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.585+0000 7f9bd3a72700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.974 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.585+0000 7f9bd3a72700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.974 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.585+0000 7f9bd3a72700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.974 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.585+0000 7f9bd3a72700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:53.974 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:08:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:08:53.585+0000 7f9bd3a72700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: Active manager daemon x restarted 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: Activating manager daemon x 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: osdmap e93: 12 total, 12 up, 12 in 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: mgrmap e22: x(active, starting, since 0.0123083s), standbys: z, y 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:08:54.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:08:54.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:08:54.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: Manager daemon x is now available 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:54 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.354+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mon.2 v2:172.21.15.87:3300/0 10 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f68700986b0 con 0x7f6878103ca0 2023-11-17T00:08:54.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.355+0000 7f686effd700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6864067120 0x7f68640674f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:54.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.355+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6878060f30 con 0x7f6864067120 2023-11-17T00:08:54.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.358+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6864067120 0x7f68640674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:54.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.362+0000 7f687db9b700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6864067120 0x7f68640674f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68780747b0 tx=0x7f6874009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:54.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: Active manager daemon x restarted 2023-11-17T00:08:54.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: Activating manager daemon x 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: osdmap e93: 12 total, 12 up, 12 in 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: mgrmap e22: x(active, starting, since 0.0123083s), standbys: z, y 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:08:54.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:08:54.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-17T00:08:54.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: Manager daemon x is now available 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:54 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.423+0000 7f686effd700 1 -- 172.21.15.6:0/2846287289 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6878060f30 con 0x7f6864067120 2023-11-17T00:08:54.432 INFO:teuthology.orchestra.run.smithi006.stdout:Schedule set for path / 2023-11-17T00:08:54.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6864067120 msgr2=0x7f68640674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6864067120 0x7f68640674f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68780747b0 tx=0x7f6874009260 comp rx=0 tx=0).stop 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878103ca0 msgr2=0x7f68781a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 --2- 172.21.15.6:0/2846287289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6878103ca0 0x7f68781a3c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f687000b6c0 tx=0x7f6870004c60 comp rx=0 tx=0).stop 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687e39c700 1 -- 172.21.15.6:0/2846287289 reap_dead start 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 shutdown_connections 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 >> 172.21.15.6:0/2846287289 conn(0x7f68780fb810 msgr2=0x7f68780fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:54.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 shutdown_connections 2023-11-17T00:08:54.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.426+0000 7f687fdff700 1 -- 172.21.15.6:0/2846287289 wait complete. 2023-11-17T00:08:54.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: Active manager daemon x restarted 2023-11-17T00:08:54.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: Activating manager daemon x 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: osdmap e93: 12 total, 12 up, 12 in 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: mgrmap e22: x(active, starting, since 0.0123083s), standbys: z, y 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-17T00:08:54.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-17T00:08:54.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: Manager daemon x is now available 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:08:54.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-17T00:08:54.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:54 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-17T00:08:54.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-11-17T00:08:54.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.664+0000 7fd6b45ff700 1 -- 172.21.15.6:0/2262504183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac10c820 msgr2=0x7fd6ac10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.664+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/2262504183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac10c820 0x7fd6ac10ec00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400b0a0 tx=0x7fd6a400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 -- 172.21.15.6:0/2262504183 shutdown_connections 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/2262504183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac074b30 0x7fd6ac072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/2262504183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac10c820 0x7fd6ac10ec00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/2262504183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 -- 172.21.15.6:0/2262504183 >> 172.21.15.6:0/2262504183 conn(0x7fd6ac0fb850 msgr2=0x7fd6ac0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 -- 172.21.15.6:0/2262504183 shutdown_connections 2023-11-17T00:08:54.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.665+0000 7fd6b45ff700 1 -- 172.21.15.6:0/2262504183 wait complete. 2023-11-17T00:08:54.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 Processor -- start 2023-11-17T00:08:54.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 -- start start 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 0x7fd6ac19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac10c820 0x7fd6ac1a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a42a0 con 0x7fd6ac10c820 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a43e0 con 0x7fd6ac074b30 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b45ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a4570 con 0x7fd6ac073550 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:54.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b1b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 0x7fd6ac19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56362/0 (socket says 172.21.15.6:56362) 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b239b700 1 -- 172.21.15.6:0/3180458281 learned_addr learned my addr 172.21.15.6:0/3180458281 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.666+0000 7fd6b2b9c700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac10c820 0x7fd6ac1a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 -- 172.21.15.6:0/3180458281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 msgr2=0x7fd6ac19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 0x7fd6ac19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 -- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac10c820 msgr2=0x7fd6ac1a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac10c820 0x7fd6ac1a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 -- 172.21.15.6:0/3180458281 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4009d00 con 0x7fd6ac073550 2023-11-17T00:08:54.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b1b9a700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 0x7fd6ac19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6b239b700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e970 tx=0x7fd6a800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.667+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8009790 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.668+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6a800c7e0 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.668+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac112230 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.668+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80054e0 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.668+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1120a0 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.669+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac102370 con 0x7fd6ac073550 2023-11-17T00:08:54.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.671+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fd6a8046080 con 0x7fd6ac073550 2023-11-17T00:08:54.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.672+0000 7fd6a37fe700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 0x7fd6980a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:54.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.672+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd6a8014070 con 0x7fd6ac073550 2023-11-17T00:08:54.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.673+0000 7fd6b1b9a700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 0x7fd6980a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:54.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.673+0000 7fd6b1b9a700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 0x7fd6980a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac074850 tx=0x7fd69c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:54.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.674+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd6a8096380 con 0x7fd6ac073550 2023-11-17T00:08:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.922+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ac061ae0 con 0x7fd6980a0fd0 2023-11-17T00:08:54.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.976+0000 7fd6a37fe700 1 -- 172.21.15.6:0/3180458281 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd6ac061ae0 con 0x7fd6980a0fd0 2023-11-17T00:08:54.978 INFO:teuthology.orchestra.run.smithi006.stdout:Retention added to path / 2023-11-17T00:08:54.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 msgr2=0x7fd6980a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 0x7fd6980a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac074850 tx=0x7fd69c009260 comp rx=0 tx=0).stop 2023-11-17T00:08:54.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 msgr2=0x7fd6ac19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:54.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e970 tx=0x7fd6a800ec80 comp rx=0 tx=0).stop 2023-11-17T00:08:54.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 shutdown_connections 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6980a0fd0 0x7fd6980a3480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ac10c820 0x7fd6ac1a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6ac073550 0x7fd6ac19c720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 --2- 172.21.15.6:0/3180458281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6ac074b30 0x7fd6ac19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 >> 172.21.15.6:0/3180458281 conn(0x7fd6ac0fb850 msgr2=0x7fd6ac10ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 shutdown_connections 2023-11-17T00:08:54.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:54.978+0000 7fd6b45ff700 1 -- 172.21.15.6:0/3180458281 wait complete. 2023-11-17T00:08:54.999 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-17T00:08:55.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 -- 172.21.15.6:0/385495597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238101470 msgr2=0x7ff2381018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/385495597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238101470 0x7ff2381018c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009990 tx=0x7ff22c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 -- 172.21.15.6:0/385495597 shutdown_connections 2023-11-17T00:08:55.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/385495597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101e00 0x7ff2381086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/385495597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238100280 0x7ff238100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/385495597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238101470 0x7ff2381018c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 -- 172.21.15.6:0/385495597 >> 172.21.15.6:0/385495597 conn(0x7ff2380fb8a0 msgr2=0x7ff2380fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:55.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.207+0000 7ff23e1ff700 1 -- 172.21.15.6:0/385495597 shutdown_connections 2023-11-17T00:08:55.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.208+0000 7ff23e1ff700 1 -- 172.21.15.6:0/385495597 wait complete. 2023-11-17T00:08:55.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.208+0000 7ff23e1ff700 1 Processor -- start 2023-11-17T00:08:55.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 -- start start 2023-11-17T00:08:55.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 0x7ff238072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238101e00 0x7ff238073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238073bc0 con 0x7ff238100280 2023-11-17T00:08:55.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238073d30 con 0x7ff238101e00 2023-11-17T00:08:55.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff23e1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2381a0070 con 0x7ff238101470 2023-11-17T00:08:55.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56946/0 (socket says 172.21.15.6:56946) 2023-11-17T00:08:55.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff2377fe700 1 -- 172.21.15.6:0/2716010708 learned_addr learned my addr 172.21.15.6:0/2716010708 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:55.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff237fff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238101e00 0x7ff238073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.209+0000 7ff236ffd700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 0x7ff238072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 -- 172.21.15.6:0/2716010708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 msgr2=0x7ff238072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 0x7ff238072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 -- 172.21.15.6:0/2716010708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238101e00 msgr2=0x7ff238073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238101e00 0x7ff238073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 -- 172.21.15.6:0/2716010708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228009520 con 0x7ff238100280 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff236ffd700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 0x7ff238072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff2377fe700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff22800ea50 tx=0x7ff22800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22800cbd0 con 0x7ff238100280 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c0095f0 con 0x7ff238100280 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff22800cd30 con 0x7ff238100280 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228018680 con 0x7ff238100280 2023-11-17T00:08:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.210+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23810bc40 con 0x7ff238100280 2023-11-17T00:08:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.212+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23810bdd0 con 0x7ff238100280 2023-11-17T00:08:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.214+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7ff228010430 con 0x7ff238100280 2023-11-17T00:08:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.215+0000 7ff234ff9700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 0x7ff2200a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.215+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff228014070 con 0x7ff238100280 2023-11-17T00:08:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.215+0000 7ff236ffd700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 0x7ff2200a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.216+0000 7ff236ffd700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 0x7ff2200a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b2e0 tx=0x7ff22c005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.219+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff2280d3080 con 0x7ff238100280 2023-11-17T00:08:55.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.473+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff238060f30 con 0x7ff2200a0f50 2023-11-17T00:08:55.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.481+0000 7ff234ff9700 1 -- 172.21.15.6:0/2716010708 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7ff238060f30 con 0x7ff2200a0f50 2023-11-17T00:08:55.483 INFO:teuthology.orchestra.run.smithi006.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-17T00:00:00", "created": "2023-11-17T00:08:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-11-17T00:08:55.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 msgr2=0x7ff2200a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 0x7ff2200a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b2e0 tx=0x7ff22c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 msgr2=0x7ff2380748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff22800ea50 tx=0x7ff22800ed60 comp rx=0 tx=0).stop 2023-11-17T00:08:55.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 shutdown_connections 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff2200a0f50 0x7ff2200a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff238100280 0x7ff2380748a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff238101e00 0x7ff238073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 --2- 172.21.15.6:0/2716010708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff238101470 0x7ff238072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.483+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 >> 172.21.15.6:0/2716010708 conn(0x7ff2380fb8a0 msgr2=0x7ff2380fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.484+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 shutdown_connections 2023-11-17T00:08:55.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.484+0000 7ff23e1ff700 1 -- 172.21.15.6:0/2716010708 wait complete. 2023-11-17T00:08:55.504 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-17T00:08:55.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: mgrmap e23: x(active, since 1.01957s), standbys: z, y 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: [17/Nov/2023:00:08:54] ENGINE Bus STARTING 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: [17/Nov/2023:00:08:54] ENGINE Serving on https://172.21.15.6:7150 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: [17/Nov/2023:00:08:54] ENGINE Serving on http://172.21.15.6:8765 2023-11-17T00:08:55.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:55 smithi186 ceph-mon[131487]: [17/Nov/2023:00:08:54] ENGINE Bus STARTED 2023-11-17T00:08:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 -- 172.21.15.6:0/2558352102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 msgr2=0x7f32d4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/2558352102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d4108710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00b0a0 tx=0x7f32cc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 -- 172.21.15.6:0/2558352102 shutdown_connections 2023-11-17T00:08:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/2558352102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d4100230 0x7f32d4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/2558352102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d4108710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/2558352102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d41014c0 0x7f32d4101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 -- 172.21.15.6:0/2558352102 >> 172.21.15.6:0/2558352102 conn(0x7f32d40fb810 msgr2=0x7f32d40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:55.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 -- 172.21.15.6:0/2558352102 shutdown_connections 2023-11-17T00:08:55.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.718+0000 7f32dc5ff700 1 -- 172.21.15.6:0/2558352102 wait complete. 2023-11-17T00:08:55.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.719+0000 7f32dc5ff700 1 Processor -- start 2023-11-17T00:08:55.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.719+0000 7f32dc5ff700 1 -- start start 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d41014c0 0x7f32d41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d419fe30 con 0x7f32d41014c0 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d419ff70 con 0x7f32d4101e70 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dc5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d41a0100 con 0x7f32d4100230 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32dab9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32d9b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d41014c0 0x7f32d41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56388/0 (socket says 172.21.15.6:56388) 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 -- 172.21.15.6:0/66170967 learned_addr learned my addr 172.21.15.6:0/66170967 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 -- 172.21.15.6:0/66170967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 msgr2=0x7f32d419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 -- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d41014c0 msgr2=0x7f32d41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d41014c0 0x7f32d41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.720+0000 7f32da39b700 1 -- 172.21.15.6:0/66170967 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc009d00 con 0x7f32d4100230 2023-11-17T00:08:55.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32da39b700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d980 tx=0x7f32d000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: mgrmap e23: x(active, since 1.01957s), standbys: z, y 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: [17/Nov/2023:00:08:54] ENGINE Bus STARTING 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: [17/Nov/2023:00:08:54] ENGINE Serving on https://172.21.15.6:7150 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: [17/Nov/2023:00:08:54] ENGINE Serving on http://172.21.15.6:8765 2023-11-17T00:08:55.755 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:55 smithi006 ceph-mon[97840]: [17/Nov/2023:00:08:54] ENGINE Bus STARTED 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0004220 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d410bc20 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32d0007640 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0005690 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.721+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d410ba90 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.722+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4067c00 con 0x7f32d4100230 2023-11-17T00:08:55.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.724+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f32d0004750 con 0x7f32d4100230 2023-11-17T00:08:55.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.726+0000 7f32cb7fe700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 0x7f32c00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:08:55.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.726+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f32d00cd370 con 0x7f32d4100230 2023-11-17T00:08:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.726+0000 7f32d9b9a700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 0x7f32c00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:08:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.727+0000 7f32d9b9a700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 0x7f32c00a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009be0 tx=0x7f32c4009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:08:55.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.731+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32d0096260 con 0x7f32d4100230 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: mgrmap e23: x(active, since 1.01957s), standbys: z, y 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: [17/Nov/2023:00:08:54] ENGINE Bus STARTING 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: [17/Nov/2023:00:08:54] ENGINE Serving on https://172.21.15.6:7150 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: [17/Nov/2023:00:08:54] ENGINE Serving on http://172.21.15.6:8765 2023-11-17T00:08:55.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:55 smithi087 ceph-mon[131182]: [17/Nov/2023:00:08:54] ENGINE Bus STARTED 2023-11-17T00:08:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.970+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f32d41a0c60 con 0x7f32c00a1350 2023-11-17T00:08:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.977+0000 7f32cb7fe700 1 -- 172.21.15.6:0/66170967 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f32d41a0c60 con 0x7f32c00a1350 2023-11-17T00:08:55.979 INFO:teuthology.orchestra.run.smithi006.stdout:/ 1M 6M3h 2023-11-17T00:08:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.979+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 msgr2=0x7f32c00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.979+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 0x7f32c00a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009be0 tx=0x7f32c4009260 comp rx=0 tx=0).stop 2023-11-17T00:08:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.979+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 msgr2=0x7f32d41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:08:55.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.979+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d980 tx=0x7f32d000dc90 comp rx=0 tx=0).stop 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 shutdown_connections 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f32c00a1350 0x7f32c00a3800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32d4101e70 0x7f32d419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32d4100230 0x7f32d41982b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 --2- 172.21.15.6:0/66170967 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32d41014c0 0x7f32d41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 >> 172.21.15.6:0/66170967 conn(0x7f32d40fb810 msgr2=0x7f32d4106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 shutdown_connections 2023-11-17T00:08:55.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:08:55.980+0000 7f32dc5ff700 1 -- 172.21.15.6:0/66170967 wait complete. 2023-11-17T00:08:56.002 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-11-17T00:08:56.080 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-11-17T00:08:56.091 INFO:teuthology.run_tasks:Running task background_exec... 2023-11-17T00:08:56.169 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:08:56.169 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-11-17T00:08:56.175 INFO:teuthology.run_tasks:Running task workunit... 2023-11-17T00:08:56.197 INFO:tasks.workunit:Pulling workunits from ref 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-17T00:08:56.197 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-17T00:08:56.197 INFO:tasks.workunit:timeout=3h 2023-11-17T00:08:56.197 INFO:tasks.workunit:cleanup=True 2023-11-17T00:08:56.197 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:56.247 INFO:teuthology.orchestra.run.smithi006.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:56.247 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-17 00:06:39.345439381 +0000 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-17 00:06:39.345439381 +0000 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-17 00:08:33.049198144 +0000 2023-11-17T00:08:56.248 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-17 00:06:39.345439381 +0000 2023-11-17T00:08:56.248 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-17T00:08:56.248 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-17T00:08:56.373 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-17T00:08:56.425 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-17T00:08:56.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-11-17T00:08:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:56 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:56.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:56.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-11-17T00:08:56.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:56.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:56 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:56.883 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: from='client.25516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:56.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:56.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-11-17T00:08:56.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:56.932 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:56 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:57.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:57 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:57 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:08:58.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:57 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:58 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:58 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:08:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:08:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:58 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:08:59.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:59 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:08:59.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:08:59 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:09:00.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:59 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:09:00.133 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:08:59 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:09:00.134 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:59 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:09:00.134 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:08:59 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:00.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:00.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:01.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:00 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:01.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:00 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:01.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:09:00.988+0000 7f9bb16ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:09:01.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:09:00.988+0000 7f9bb16ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:09:01.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:09:00.988+0000 7f9bb16ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:09:01.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:09:00.988+0000 7f9bb16ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:09:01.473 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:09:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:09:00.988+0000 7f9bb16ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:09:01.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:01.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 132 KiB/s wr, 47 op/s 2023-11-17T00:09:01.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:01 smithi006 ceph-mon[97840]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 132 KiB/s wr, 47 op/s 2023-11-17T00:09:02.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:01 smithi087 ceph-mon[131182]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi006:/etc/ceph/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi087:/etc/ceph/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi186:/etc/ceph/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 132 KiB/s wr, 47 op/s 2023-11-17T00:09:02.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:01 smithi186 ceph-mon[131487]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2023-11-17T00:09:03.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:03.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: mgrmap e25: x(active, since 8s), standbys: z, y 2023-11-17T00:09:03.601 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:03.602 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:03 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: mgrmap e25: x(active, since 8s), standbys: z, y 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:03.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:03 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: Updating smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: mgrmap e25: x(active, since 8s), standbys: z, y 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:09:03.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:03 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:09:04.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:04 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.820 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:04 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:04.820 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:04 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:09:04.820 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:04 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.821 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:04 smithi006 ceph-mon[97840]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 102 KiB/s wr, 37 op/s 2023-11-17T00:09:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:04 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:04 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:04 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:09:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:04 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:04 smithi087 ceph-mon[131182]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 102 KiB/s wr, 37 op/s 2023-11-17T00:09:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:04 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:04 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:09:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:04 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:09:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:04 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:09:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:04 smithi186 ceph-mon[131487]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 102 KiB/s wr, 37 op/s 2023-11-17T00:09:06.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:06 smithi006 ceph-mon[97840]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 108 KiB/s wr, 44 op/s 2023-11-17T00:09:06.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:06 smithi087 ceph-mon[131182]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 108 KiB/s wr, 44 op/s 2023-11-17T00:09:06.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:06 smithi186 ceph-mon[131487]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 108 KiB/s wr, 44 op/s 2023-11-17T00:09:08.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:08 smithi006 ceph-mon[97840]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:08.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:08 smithi087 ceph-mon[131182]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:08.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:08 smithi186 ceph-mon[131487]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 -- 172.21.15.6:0/1840693856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7481004a0 msgr2=0x7fc748100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1840693856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7481004a0 0x7fc748100870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc744009990 tx=0x7fc744009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 -- 172.21.15.6:0/1840693856 shutdown_connections 2023-11-17T00:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1840693856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481054e0 0x7fc74810d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1840693856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7481004a0 0x7fc748100870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1840693856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc748100e40 0x7fc748104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 -- 172.21.15.6:0/1840693856 >> 172.21.15.6:0/1840693856 conn(0x7fc7480fbd10 msgr2=0x7fc7480fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 -- 172.21.15.6:0/1840693856 shutdown_connections 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.436+0000 7fc7503ff700 1 -- 172.21.15.6:0/1840693856 wait complete. 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.437+0000 7fc7503ff700 1 Processor -- start 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.437+0000 7fc7503ff700 1 -- start start 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.437+0000 7fc7503ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc7503ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 0x7fc748198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc7503ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7481054e0 0x7fc74819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc7503ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819d600 con 0x7fc7481004a0 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc7503ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819d770 con 0x7fc748100e40 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc7503ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819d930 con 0x7fc7481054e0 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58304/0 (socket says 172.21.15.6:58304) 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 -- 172.21.15.6:0/1747277632 learned_addr learned my addr 172.21.15.6:0/1747277632 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e99c700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7481054e0 0x7fc74819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74d99a700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 0x7fc748198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 -- 172.21.15.6:0/1747277632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7481054e0 msgr2=0x7fc74819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7481054e0 0x7fc74819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 -- 172.21.15.6:0/1747277632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 msgr2=0x7fc748198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 0x7fc748198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74e19b700 1 -- 172.21.15.6:0/1747277632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440095f0 con 0x7fc7481004a0 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.438+0000 7fc74d99a700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 0x7fc748198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc74e19b700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc744005910 tx=0x7fc744005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc73f7fe700 1 -- 172.21.15.6:0/1747277632 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74401d070 con 0x7fc7481004a0 2023-11-17T00:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc73f7fe700 1 -- 172.21.15.6:0/1747277632 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc744022470 con 0x7fc7481004a0 2023-11-17T00:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7481102f0 con 0x7fc7481004a0 2023-11-17T00:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc73f7fe700 1 -- 172.21.15.6:0/1747277632 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744017650 con 0x7fc7481004a0 2023-11-17T00:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.439+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc748110160 con 0x7fc7481004a0 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.442+0000 7fc73f7fe700 1 -- 172.21.15.6:0/1747277632 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fc7440225e0 con 0x7fc7481004a0 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.444+0000 7fc73f7fe700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 0x7fc7340a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.444+0000 7fc73f7fe700 1 -- 172.21.15.6:0/1747277632 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7440ce740 con 0x7fc7481004a0 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.444+0000 7fc74d99a700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 0x7fc7340a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.444+0000 7fc763cbab80 1 Processor -- start 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.444+0000 7fc763cbab80 1 -- start start 2023-11-17T00:09:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74d99a700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 0x7fc7340a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc748199c30 tx=0x7fc738009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bd7fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bc85000 con 0x55884bd85720 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bcd7840 con 0x55884bbdb6b0 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc763cbab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bcd0cd0 con 0x55884bd85b90 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74d99a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74d99a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58320/0 (socket says 172.21.15.6:58320) 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bd7fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.445+0000 7fc74d99a700 1 -- 172.21.15.6:0/3593591171 learned_addr learned my addr 172.21.15.6:0/3593591171 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 -- 172.21.15.6:0/3593591171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 msgr2=0x55884bd7fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bd7fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 -- 172.21.15.6:0/3593591171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 msgr2=0x55884bd7f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 -- 172.21.15.6:0/3593591171 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884bb525d0 con 0x55884bbdb6b0 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74d99a700 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd851e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7440cb960 tx=0x7fc744009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e314cc7348feaa server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc73cbfe700 1 -- 172.21.15.6:0/3593591171 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74409ac00 con 0x55884bbdb6b0 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc73cbfe700 1 -- 172.21.15.6:0/3593591171 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc74409b2a0 con 0x55884bbdb6b0 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.446+0000 7fc73cbfe700 1 -- 172.21.15.6:0/3593591171 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440ce330 con 0x55884bbdb6b0 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 -- 172.21.15.6:0/3593591171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 msgr2=0x55884bd851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd851e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7440cb960 tx=0x7fc744009310 comp rx=0 tx=0).stop 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 -- 172.21.15.6:0/3593591171 shutdown_connections 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bd7fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd851e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3593591171 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 -- 172.21.15.6:0/3593591171 >> 172.21.15.6:0/3593591171 conn(0x55884bd74f50 msgr2=0x55884bd7a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 -- 172.21.15.6:0/3593591171 shutdown_connections 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.447+0000 7fc763cbab80 1 -- 172.21.15.6:0/3593591171 wait complete. 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.448+0000 7fc763cbab80 1 -- start start 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.448+0000 7fc763cbab80 1 Processor -- start 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85720 0x55884bd8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bc72880 con 0x55884bd85b90 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bc72520 con 0x55884bbdb6b0 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bb548c0 con 0x55884bd85720 2023-11-17T00:09:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc763cbab80 10 client.0 _ll_register_callbacks cb 0x55884bc90720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34194/0 (socket says 172.21.15.6:34194) 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.449+0000 7fc74e19b700 1 -- 172.21.15.6:0/3341139185 learned_addr learned my addr 172.21.15.6:0/3341139185 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e99c700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74d99a700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85720 0x55884bd8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 -- 172.21.15.6:0/3341139185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85720 msgr2=0x55884bd8ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85720 0x55884bd8ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 -- 172.21.15.6:0/3341139185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 msgr2=0x55884bd88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 -- 172.21.15.6:0/3341139185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc748110160 con 0x55884bbdb6b0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc74e19b700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f740 tx=0x7fc74400b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6478f510f094cb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc731ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440b2070 con 0x55884bbdb6b0 2023-11-17T00:09:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc74409f2d0 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.450+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55884bd72710 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc74409ac00 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 10 client.25562 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55884bc97130 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55884bd72710 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7440a32f0 con 0x55884bbdb6b0 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 10 client.25562 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 20 client.25562 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bd98760 0x55884bd9ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 4 client.25562 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.451+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55884bc93040 con 0x55884bd98760 2023-11-17T00:09:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.452+0000 7fc74d99a700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bd98760 0x55884bd9ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.452+0000 7fc731ffb700 10 client.25562 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.452+0000 7fc74d99a700 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bd98760 0x55884bd9ab40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69201daed27a9404 server_cookie=53da80228cbc0e15 in_seq=0 out_seq=0 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc731ffb700 1 -- 172.21.15.6:0/3341139185 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55884bc93040 con 0x55884bd98760 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc731ffb700 10 client.25562 handle_command_reply: tid=0 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 client.25562 shutdown 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 msgr2=0x55884bd8e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f740 tx=0x7fc74400b490 comp rx=0 tx=0).stop 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 shutdown_connections 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bd98760 0x55884bd9ab40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8e7e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 --2- 172.21.15.6:0/3341139185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85720 0x55884bd8ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 >> 172.21.15.6:0/3341139185 conn(0x55884bd74f50 msgr2=0x55884bd797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc731ffb700 0 client.25562 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 shutdown_connections 2023-11-17T00:09:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.453+0000 7fc763cbab80 1 -- 172.21.15.6:0/3341139185 wait complete. 2023-11-17T00:09:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.454+0000 7fc763cbab80 20 client.25562 trim_cache size 0 max 16384 2023-11-17T00:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 Processor -- start 2023-11-17T00:09:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 -- start start 2023-11-17T00:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bbdb6b0 0x55884bd88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bd74b70 con 0x55884bd85b90 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bd7bba0 con 0x55884bd85720 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.472+0000 7fc763cbab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bd768c0 con 0x55884bbdb6b0 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74e99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bbdb6b0 0x55884bd88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34206/0 (socket says 172.21.15.6:34206) 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74e99c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd81c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58338/0 (socket says 172.21.15.6:58338) 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 -- 172.21.15.6:0/2246903339 learned_addr learned my addr 172.21.15.6:0/2246903339 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 -- 172.21.15.6:0/2246903339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bbdb6b0 msgr2=0x55884bd88a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bbdb6b0 0x55884bd88a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 -- 172.21.15.6:0/2246903339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 msgr2=0x55884bd81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd81c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 -- 172.21.15.6:0/2246903339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884bc96e80 con 0x55884bd85720 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.473+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc738009fd0 tx=0x7fc7380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8a93f607c64ac3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc73d3ff700 1 -- 172.21.15.6:0/2246903339 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738025040 con 0x55884bd85720 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc73d3ff700 1 -- 172.21.15.6:0/2246903339 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7380044f0 con 0x55884bd85720 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc73d3ff700 1 -- 172.21.15.6:0/2246903339 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738012450 con 0x55884bd85720 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 -- 172.21.15.6:0/2246903339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 msgr2=0x55884bd79f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc738009fd0 tx=0x7fc7380047c0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 -- 172.21.15.6:0/2246903339 shutdown_connections 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85b90 0x55884bd81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bbdb6b0 0x55884bd88a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2246903339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bd85720 0x55884bd79f60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 -- 172.21.15.6:0/2246903339 >> 172.21.15.6:0/2246903339 conn(0x55884bd74f50 msgr2=0x55884bd88360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.474+0000 7fc763cbab80 1 -- 172.21.15.6:0/2246903339 shutdown_connections 2023-11-17T00:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.475+0000 7fc763cbab80 1 -- 172.21.15.6:0/2246903339 wait complete. 2023-11-17T00:09:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.475+0000 7fc763cbab80 1 -- start start 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 Processor -- start 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bdeb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bd8d340 con 0x55884bd85720 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bdeba90 con 0x55884bbdb6b0 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.476+0000 7fc763cbab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884bdebbd0 con 0x55884bd85b90 2023-11-17T00:09:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc763cbab80 10 client.0 _ll_register_callbacks cb 0x7fc738016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74d99a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34218/0 (socket says 172.21.15.6:34218) 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 -- 172.21.15.6:0/2302341139 learned_addr learned my addr 172.21.15.6:0/2302341139 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e99c700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bdeb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 -- 172.21.15.6:0/2302341139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 msgr2=0x55884bdeb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bdeb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 -- 172.21.15.6:0/2302341139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 msgr2=0x55884bd8ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd8ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74e19b700 1 -- 172.21.15.6:0/2302341139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55884bae7c00 con 0x55884bbdb6b0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.477+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd8ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc74e19b700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7440a3130 tx=0x7fc7440cbaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33184d2eb486144c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440b2060 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc74409b2a0 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7440a3660 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc744084aa0 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc763cbab80 10 client.25568 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55884bafed80 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc74409b2a0 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc744097090 con 0x55884bbdb6b0 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc763cbab80 10 client.25568 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.478+0000 7fc763cbab80 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc763cbab80 20 client.25568 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bdecda0 0x55884bdef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc763cbab80 4 client.25568 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55884bc9f600 con 0x55884bdecda0 2023-11-17T00:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bdecda0 0x55884bdef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc74d99a700 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bdecda0 0x55884bdef180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60f09d892660e24 server_cookie=abc2d8a5db324f5c in_seq=0 out_seq=0 2023-11-17T00:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.479+0000 7fc731ffb700 10 client.25568 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc731ffb700 1 -- 172.21.15.6:0/2302341139 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55884bc9f600 con 0x55884bdecda0 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc731ffb700 10 client.25568 handle_command_reply: tid=0 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 client.25568 shutdown 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 msgr2=0x55884bd8c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc7440a3130 tx=0x7fc7440cbaa0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 shutdown_connections 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55884bd85b90 0x55884bdeb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55884bbdb6b0 0x55884bd8c540 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55884bdecda0 0x55884bdef180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 --2- 172.21.15.6:0/2302341139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55884bd85720 0x55884bd8ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc731ffb700 0 client.25568 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.494+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 >> 172.21.15.6:0/2302341139 conn(0x55884bd74f50 msgr2=0x55884bd84430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 shutdown_connections 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc763cbab80 1 -- 172.21.15.6:0/2302341139 wait complete. 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc763cbab80 20 client.25568 trim_cache size 0 max 16384 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:09:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:09:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:09:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 msgr2=0x7fc7340a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 0x7fc7340a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc748199c30 tx=0x7fc738009260 comp rx=0 tx=0).stop 2023-11-17T00:09:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 msgr2=0x7fc748198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.495+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc744005910 tx=0x7fc744005720 comp rx=0 tx=0).stop 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 shutdown_connections 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc7340a13e0 0x7fc7340a3890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7481004a0 0x7fc748198620 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7481054e0 0x7fc74819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 --2- 172.21.15.6:0/1747277632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc748100e40 0x7fc748198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 >> 172.21.15.6:0/1747277632 conn(0x7fc7480fbd10 msgr2=0x7fc7480fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 shutdown_connections 2023-11-17T00:09:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:09.496+0000 7fc7503ff700 1 -- 172.21.15.6:0/1747277632 wait complete. 2023-11-17T00:09:10.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:10 smithi006 ceph-mon[97840]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 100 KiB/s wr, 42 op/s 2023-11-17T00:09:10.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:10 smithi087 ceph-mon[131182]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 100 KiB/s wr, 42 op/s 2023-11-17T00:09:10.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:10 smithi186 ceph-mon[131487]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 100 KiB/s wr, 42 op/s 2023-11-17T00:09:12.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:12 smithi006 ceph-mon[97840]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:12.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:12 smithi087 ceph-mon[131182]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:12.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:12 smithi186 ceph-mon[131487]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 99 KiB/s wr, 41 op/s 2023-11-17T00:09:14.035 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 -- 172.21.15.6:0/3131045634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 msgr2=0x7f54b010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 --2- 172.21.15.6:0/3131045634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b010f230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009f90 tx=0x7f54a800b330 comp rx=0 tx=0).stop 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 -- 172.21.15.6:0/3131045634 shutdown_connections 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 --2- 172.21.15.6:0/3131045634 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b0072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 --2- 172.21.15.6:0/3131045634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b010f230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 --2- 172.21.15.6:0/3131045634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 0x7f54b0073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 -- 172.21.15.6:0/3131045634 >> 172.21.15.6:0/3131045634 conn(0x7f54b0077500 msgr2=0x7f54b0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 -- 172.21.15.6:0/3131045634 shutdown_connections 2023-11-17T00:09:14.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.384+0000 7f54b59ff700 1 -- 172.21.15.6:0/3131045634 wait complete. 2023-11-17T00:09:14.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.386+0000 7f54b59ff700 1 Processor -- start 2023-11-17T00:09:14.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.386+0000 7f54b59ff700 1 -- start start 2023-11-17T00:09:14.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 0x7f54b0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019c5d0 con 0x7f54b00746a0 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019c740 con 0x7f54b0074b00 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54b59ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019c900 con 0x7f54b010ce50 2023-11-17T00:09:14.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58354/0 (socket says 172.21.15.6:58354) 2023-11-17T00:09:14.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54aeffd700 1 -- 172.21.15.6:0/1759016991 learned_addr learned my addr 172.21.15.6:0/1759016991 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.387+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 0x7f54b0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 -- 172.21.15.6:0/1759016991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 msgr2=0x7f54b0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 0x7f54b0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 -- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 msgr2=0x7f54b01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 -- 172.21.15.6:0/1759016991 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8009c40 con 0x7f54b010ce50 2023-11-17T00:09:14.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.388+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b019bfd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b6c0 tx=0x7f54a80094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.389+0000 7f54b49fd700 1 -- 172.21.15.6:0/1759016991 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800e040 con 0x7f54b010ce50 2023-11-17T00:09:14.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.389+0000 7f54b49fd700 1 -- 172.21.15.6:0/1759016991 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a8004e70 con 0x7f54b010ce50 2023-11-17T00:09:14.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.389+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b0112520 con 0x7f54b010ce50 2023-11-17T00:09:14.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.389+0000 7f54b49fd700 1 -- 172.21.15.6:0/1759016991 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8005040 con 0x7f54b010ce50 2023-11-17T00:09:14.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.389+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b0112390 con 0x7f54b010ce50 2023-11-17T00:09:14.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.394+0000 7f54b49fd700 1 -- 172.21.15.6:0/1759016991 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f54a801d470 con 0x7f54b010ce50 2023-11-17T00:09:14.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.395+0000 7f54c938eb80 1 Processor -- start 2023-11-17T00:09:14.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 -- start start 2023-11-17T00:09:14.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca13f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca188b0 0x55c9aca18d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac919060 con 0x55c9aca13b60 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac96b8a0 con 0x55c9aca14470 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54c938eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac964d30 con 0x55c9aca188b0 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.396+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.397+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca18370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34240/0 (socket says 172.21.15.6:34240) 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.397+0000 7f54aeffd700 1 -- 172.21.15.6:0/3874583656 learned_addr learned my addr 172.21.15.6:0/3874583656 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.397+0000 7f54b49fd700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 0x7f54980a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.397+0000 7f54b49fd700 1 -- 172.21.15.6:0/1759016991 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f54a80ce5c0 con 0x7f54b010ce50 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.397+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 0x7f54980a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca188b0 0x55c9aca18d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca13f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54af7fe700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 0x7f54980a3910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54a80cb840 tx=0x7f54a801df90 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 -- 172.21.15.6:0/3874583656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca188b0 msgr2=0x55c9aca18d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca188b0 0x55c9aca18d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 -- 172.21.15.6:0/3874583656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 msgr2=0x55c9aca18370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca18370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.398+0000 7f54ae7fc700 1 -- 172.21.15.6:0/3874583656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9ac7e6630 con 0x55c9aca13b60 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.399+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca13f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54b0199df0 tx=0x7f54a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f999d109dc9371c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.399+0000 7f5495ffb700 1 -- 172.21.15.6:0/3874583656 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a401f600 con 0x55c9aca13b60 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.399+0000 7f5495ffb700 1 -- 172.21.15.6:0/3874583656 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a400f880 con 0x55c9aca13b60 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.399+0000 7f5495ffb700 1 -- 172.21.15.6:0/3874583656 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a401e610 con 0x55c9aca13b60 2023-11-17T00:09:14.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 -- 172.21.15.6:0/3874583656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 msgr2=0x55c9aca13f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca13f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54b0199df0 tx=0x7f54a4009310 comp rx=0 tx=0).stop 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 -- 172.21.15.6:0/3874583656 shutdown_connections 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca13f30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca188b0 0x55c9aca18d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3874583656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca18370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.400+0000 7f54c938eb80 1 -- 172.21.15.6:0/3874583656 >> 172.21.15.6:0/3874583656 conn(0x55c9aca09360 msgr2=0x55c9aca0eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.401+0000 7f54c938eb80 1 -- 172.21.15.6:0/3874583656 shutdown_connections 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.401+0000 7f54c938eb80 1 -- 172.21.15.6:0/3874583656 wait complete. 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.401+0000 7f54c938eb80 1 -- start start 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.401+0000 7f54c938eb80 1 Processor -- start 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca13b60 0x55c9aca20000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac9068e0 con 0x55c9aca188b0 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac906580 con 0x55c9aca13b60 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ac7e8920 con 0x55c9aca14470 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54c938eb80 10 client.0 _ll_register_callbacks cb 0x55c9ac924780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.402+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca13b60 0x55c9aca20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43942/0 (socket says 172.21.15.6:43942) 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 -- 172.21.15.6:0/3690766779 learned_addr learned my addr 172.21.15.6:0/3690766779 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 -- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca13b60 msgr2=0x55c9aca20000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca13b60 0x55c9aca20000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 -- 172.21.15.6:0/3690766779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 msgr2=0x55c9aca1d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca1d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 -- 172.21.15.6:0/3690766779 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a40105e0 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.403+0000 7f54aeffd700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e800 tx=0x7f54a000eb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e24f8e2a230206 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.404+0000 7f548effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.404+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a001e070 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.404+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f54a000d670 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.404+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9aca06b20 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a000d830 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f54c938eb80 10 client.25564 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c9ac92b190 con 0x55c9aca14470 2023-11-17T00:09:14.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9aca06b20 con 0x55c9aca14470 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54a002dbe0 con 0x55c9aca14470 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f54c938eb80 10 client.25564 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f54c938eb80 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.405+0000 7f54c938eb80 20 client.25564 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.406+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca2d850 0x55c9aca2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.406+0000 7f54c938eb80 4 client.25564 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.406+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9ac9270a0 con 0x55c9aca2d850 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.406+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca2d850 0x55c9aca2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.407+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca2d850 0x55c9aca2fc60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdaf0a9a6709f683 server_cookie=e50a7b161207936f in_seq=0 out_seq=0 2023-11-17T00:09:14.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.407+0000 7f548effd700 10 client.25564 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:14.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.407+0000 7f548effd700 1 -- 172.21.15.6:0/3690766779 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9ac9270a0 con 0x55c9aca2d850 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.407+0000 7f548effd700 10 client.25564 handle_command_reply: tid=0 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 client.25564 shutdown 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 msgr2=0x55c9aca1d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e800 tx=0x7f54a000eb10 comp rx=0 tx=0).stop 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 shutdown_connections 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca2d850 0x55c9aca2fc60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca13b60 0x55c9aca20000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 --2- 172.21.15.6:0/3690766779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca1d280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 >> 172.21.15.6:0/3690766779 conn(0x55c9aca09360 msgr2=0x55c9aca0dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f548effd700 0 client.25564 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 shutdown_connections 2023-11-17T00:09:14.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.408+0000 7f54c938eb80 1 -- 172.21.15.6:0/3690766779 wait complete. 2023-11-17T00:09:14.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.409+0000 7f54c938eb80 20 client.25564 trim_cache size 0 max 16384 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 Processor -- start 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 -- start start 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 0x55c9aca179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca0e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca08ee0 con 0x55c9aca188b0 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca0f950 con 0x55c9aca14470 2023-11-17T00:09:14.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.443+0000 7f54c938eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca1b7e0 con 0x55c9aca13b60 2023-11-17T00:09:14.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34256/0 (socket says 172.21.15.6:34256) 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 -- 172.21.15.6:0/1989008004 learned_addr learned my addr 172.21.15.6:0/1989008004 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 0x55c9aca179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca0e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 -- 172.21.15.6:0/1989008004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 msgr2=0x55c9aca179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 0x55c9aca179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 -- 172.21.15.6:0/1989008004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 msgr2=0x55c9aca0e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca0e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54aeffd700 1 -- 172.21.15.6:0/1989008004 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9ac92aee0 con 0x55c9aca14470 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.444+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 0x55c9aca179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.445+0000 7f54aeffd700 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009a50 tx=0x7f54a00191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1956f414d4bf54 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.445+0000 7f54967fc700 1 -- 172.21.15.6:0/1989008004 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0032070 con 0x55c9aca14470 2023-11-17T00:09:14.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.445+0000 7f54967fc700 1 -- 172.21.15.6:0/1989008004 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a001f040 con 0x55c9aca14470 2023-11-17T00:09:14.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.445+0000 7f54967fc700 1 -- 172.21.15.6:0/1989008004 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a001e030 con 0x55c9aca14470 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 -- 172.21.15.6:0/1989008004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 msgr2=0x55c9aca17f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009a50 tx=0x7f54a00191a0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 -- 172.21.15.6:0/1989008004 shutdown_connections 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca188b0 0x55c9aca0e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca13b60 0x55c9aca179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 --2- 172.21.15.6:0/1989008004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca14470 0x55c9aca17f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 -- 172.21.15.6:0/1989008004 >> 172.21.15.6:0/1989008004 conn(0x55c9aca09360 msgr2=0x55c9aca29be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 -- 172.21.15.6:0/1989008004 shutdown_connections 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.446+0000 7f54c938eb80 1 -- 172.21.15.6:0/1989008004 wait complete. 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.447+0000 7f54c938eb80 1 -- start start 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.447+0000 7f54c938eb80 1 Processor -- start 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca188b0 0x55c9aca212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca219c0 con 0x55c9aca13b60 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca21b30 con 0x55c9aca188b0 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9aca81710 con 0x55c9aca14470 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54c938eb80 10 client.0 _ll_register_callbacks cb 0x7f54a0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:14.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58378/0 (socket says 172.21.15.6:58378) 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54ae7fc700 1 -- 172.21.15.6:0/2576356099 learned_addr learned my addr 172.21.15.6:0/2576356099 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca188b0 0x55c9aca212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.448+0000 7f54aeffd700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 -- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca188b0 msgr2=0x55c9aca212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca188b0 0x55c9aca212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 -- 172.21.15.6:0/2576356099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 msgr2=0x55c9aca24a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 -- 172.21.15.6:0/2576356099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9ac92b190 con 0x55c9aca14470 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f54aeffd700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca20d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096b0 tx=0x7f54a00196e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0020d37ed5f4283 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.449+0000 7f548effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.450+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0032040 con 0x55c9aca14470 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.450+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f54a0000e40 con 0x55c9aca14470 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.450+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54a400f880 con 0x55c9aca14470 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a001e040 con 0x55c9aca14470 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f54c938eb80 10 client.25570 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c9ac77bc60 con 0x55c9aca14470 2023-11-17T00:09:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a400f880 con 0x55c9aca14470 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54a000d670 con 0x55c9aca14470 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f54c938eb80 10 client.25570 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f54c938eb80 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.451+0000 7f54c938eb80 20 client.25570 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.452+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca82a10 0x55c9aca84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.452+0000 7f54c938eb80 4 client.25570 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.452+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9ac933660 con 0x55c9aca82a10 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.452+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca82a10 0x55c9aca84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f54ae7fc700 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca82a10 0x55c9aca84e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b947198f70a49d38 server_cookie=5208f5decc698a35 in_seq=0 out_seq=0 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f548effd700 10 client.25570 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f548effd700 1 -- 172.21.15.6:0/2576356099 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c9ac933660 con 0x55c9aca82a10 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f548effd700 10 client.25570 handle_command_reply: tid=0 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f54c938eb80 1 client.25570 shutdown 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 msgr2=0x55c9aca20d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.453+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca20d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54a00096b0 tx=0x7f54a00196e0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 shutdown_connections 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c9aca82a10 0x55c9aca84e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c9aca13b60 0x55c9aca24a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c9aca188b0 0x55c9aca212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f548effd700 0 client.25570 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 --2- 172.21.15.6:0/2576356099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c9aca14470 0x55c9aca20d70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 >> 172.21.15.6:0/2576356099 conn(0x55c9aca09360 msgr2=0x55c9aca179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 shutdown_connections 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 1 -- 172.21.15.6:0/2576356099 wait complete. 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.454+0000 7f54c938eb80 20 client.25570 trim_cache size 0 max 16384 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:09:14.473 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 msgr2=0x7f54980a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 0x7f54980a3910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54a80cb840 tx=0x7f54a801df90 comp rx=0 tx=0).stop 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 msgr2=0x7f54b019bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b019bfd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b6c0 tx=0x7f54a80094b0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 shutdown_connections 2023-11-17T00:09:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f54980a1460 0x7f54980a3910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54b00746a0 0x7f54b01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b010ce50 0x7f54b019bfd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 --2- 172.21.15.6:0/1759016991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54b0074b00 0x7f54b0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 >> 172.21.15.6:0/1759016991 conn(0x7f54b0077500 msgr2=0x7f54b0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 shutdown_connections 2023-11-17T00:09:14.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.455+0000 7f54b59ff700 1 -- 172.21.15.6:0/1759016991 wait complete. 2023-11-17T00:09:14.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:09:14.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 -- 172.21.15.6:0/1817087824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 msgr2=0x7f382c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 --2- 172.21.15.6:0/1817087824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c104ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 -- 172.21.15.6:0/1817087824 shutdown_connections 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 --2- 172.21.15.6:0/1817087824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 0x7f382c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 --2- 172.21.15.6:0/1817087824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c10d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 --2- 172.21.15.6:0/1817087824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c104ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 -- 172.21.15.6:0/1817087824 >> 172.21.15.6:0/1817087824 conn(0x7f382c0774d0 msgr2=0x7f382c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 -- 172.21.15.6:0/1817087824 shutdown_connections 2023-11-17T00:09:14.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.719+0000 7f3833bff700 1 -- 172.21.15.6:0/1817087824 wait complete. 2023-11-17T00:09:14.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.721+0000 7f3833bff700 1 Processor -- start 2023-11-17T00:09:14.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.721+0000 7f3833bff700 1 -- start start 2023-11-17T00:09:14.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:14 smithi006 ceph-mon[97840]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-17T00:09:14.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.721+0000 7f3833bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 0x7f382c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f3833bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f3833bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f3833bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19b460 con 0x7f382c100cc0 2023-11-17T00:09:14.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f3833bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19b5d0 con 0x7f382c105620 2023-11-17T00:09:14.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f3833bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c19b790 con 0x7f382c101660 2023-11-17T00:09:14.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43976/0 (socket says 172.21.15.6:43976) 2023-11-17T00:09:14.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 -- 172.21.15.6:0/2430344444 learned_addr learned my addr 172.21.15.6:0/2430344444 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383199b700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 0x7f382c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383219c700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 -- 172.21.15.6:0/2430344444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 msgr2=0x7f382c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 -- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 msgr2=0x7f382c196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.722+0000 7f383119a700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 0x7f382c196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f383119a700 1 -- 172.21.15.6:0/2430344444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0095f0 con 0x7f382c101660 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f383219c700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f383119a700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f381c015000 tx=0x7f381c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c01d070 con 0x7f382c101660 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f381c021410 con 0x7f382c101660 2023-11-17T00:09:14.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c110370 con 0x7f382c101660 2023-11-17T00:09:14.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.723+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c017650 con 0x7f382c101660 2023-11-17T00:09:14.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.724+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c1101e0 con 0x7f382c101660 2023-11-17T00:09:14.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.725+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c110500 con 0x7f382c101660 2023-11-17T00:09:14.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.727+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f381c021810 con 0x7f382c101660 2023-11-17T00:09:14.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.728+0000 7f3822ffd700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 0x7f38180a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:14.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.728+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f381c0ce6e0 con 0x7f382c101660 2023-11-17T00:09:14.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.729+0000 7f383199b700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 0x7f38180a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:14.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.730+0000 7f383199b700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 0x7f38180a3910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3828009c60 tx=0x7f3828009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:14.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.731+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f381c097580 con 0x7f382c101660 2023-11-17T00:09:14.855 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:14 smithi087 ceph-mon[131182]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-17T00:09:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:14 smithi186 ceph-mon[131487]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-17T00:09:14.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.991+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382c067af0 con 0x7f382c101660 2023-11-17T00:09:14.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.992+0000 7f3822ffd700 1 -- 172.21.15.6:0/2430344444 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f381c09aba0 con 0x7f382c101660 2023-11-17T00:09:14.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:09:14.995 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 msgr2=0x7f38180a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 0x7f38180a3910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3828009c60 tx=0x7f3828009260 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 msgr2=0x7f382c1969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f381c015000 tx=0x7f381c009580 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 shutdown_connections 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f38180a1460 0x7f38180a3910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f382c100cc0 0x7f382c196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c105620 0x7f382c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 --2- 172.21.15.6:0/2430344444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f382c101660 0x7f382c1969c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 >> 172.21.15.6:0/2430344444 conn(0x7f382c0774d0 msgr2=0x7f382c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 shutdown_connections 2023-11-17T00:09:15.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:14.995+0000 7f3833bff700 1 -- 172.21.15.6:0/2430344444 wait complete. 2023-11-17T00:09:15.001 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:09:15.017 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:09:15.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/3189401372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 msgr2=0x7f3bb4106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/3189401372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb4106450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009990 tx=0x7f3ba4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/3189401372 shutdown_connections 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/3189401372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/3189401372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb4106450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/3189401372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb4068210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.240+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/3189401372 >> 172.21.15.6:0/3189401372 conn(0x7f3bb4074ef0 msgr2=0x7f3bb40752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.241+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/3189401372 shutdown_connections 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.241+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/3189401372 wait complete. 2023-11-17T00:09:15.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.242+0000 7f3bb8b9d700 1 Processor -- start 2023-11-17T00:09:15.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.242+0000 7f3bb8b9d700 1 -- start start 2023-11-17T00:09:15.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb40ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb40ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb41a8b80 con 0x7f3bb4068750 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb41a8cc0 con 0x7f3bb4106080 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb8b9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb41a8e00 con 0x7f3bb4067dc0 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb40ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58400/0 (socket says 172.21.15.6:58400) 2023-11-17T00:09:15.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.243+0000 7f3bb2ffd700 1 -- 172.21.15.6:0/21697692 learned_addr learned my addr 172.21.15.6:0/21697692 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb40ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 -- 172.21.15.6:0/21697692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 msgr2=0x7f3bb40ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb40ffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 -- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 msgr2=0x7f3bb40ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 -- 172.21.15.6:0/21697692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40095f0 con 0x7f3bb4067dc0 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.244+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb40ff160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b2e0 tx=0x7f3ba4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.245+0000 7f3bb0ff9700 1 -- 172.21.15.6:0/21697692 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba401d070 con 0x7f3bb4067dc0 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.245+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb4110070 con 0x7f3bb4067dc0 2023-11-17T00:09:15.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.245+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb410fee0 con 0x7f3bb4067dc0 2023-11-17T00:09:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.246+0000 7f3bb0ff9700 1 -- 172.21.15.6:0/21697692 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba4005030 con 0x7f3bb4067dc0 2023-11-17T00:09:15.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.246+0000 7f3bb0ff9700 1 -- 172.21.15.6:0/21697692 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004a00 con 0x7f3bb4067dc0 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.249+0000 7f3bcd808b80 1 Processor -- start 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 -- start start 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bb0ff9700 1 -- 172.21.15.6:0/21697692 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3ba4004b60 con 0x7f3bb4067dc0 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d5693060 con 0x55a5d57a3ab0 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d56e58a0 con 0x55a5d579f6d0 2023-11-17T00:09:15.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.250+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d56ded30 con 0x55a5d579edc0 2023-11-17T00:09:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43992/0 (socket says 172.21.15.6:43992) 2023-11-17T00:09:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 -- 172.21.15.6:0/1475957882 learned_addr learned my addr 172.21.15.6:0/1475957882 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:15.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 -- 172.21.15.6:0/1475957882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 msgr2=0x55a5d57a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 -- 172.21.15.6:0/1475957882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 msgr2=0x55a5d57a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.251+0000 7f3bb3fff700 1 -- 172.21.15.6:0/1475957882 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5d5560630 con 0x55a5d579edc0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bac003eb0 tx=0x7f3bac009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd0006f7a5ca0b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3b99ffb700 1 -- 172.21.15.6:0/1475957882 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0092d0 con 0x55a5d579edc0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3b99ffb700 1 -- 172.21.15.6:0/1475957882 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac00f040 con 0x55a5d579edc0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3b99ffb700 1 -- 172.21.15.6:0/1475957882 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00e8e0 con 0x55a5d579edc0 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.252+0000 7f3bb0ff9700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 0x7f3b9c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bb0ff9700 1 -- 172.21.15.6:0/21697692 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ba40ce170 con 0x7f3bb4067dc0 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 0x7f3b9c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 -- 172.21.15.6:0/1475957882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 msgr2=0x55a5d579f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bac003eb0 tx=0x7f3bac009920 comp rx=0 tx=0).stop 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 -- 172.21.15.6:0/1475957882 shutdown_connections 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d579f190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/1475957882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 -- 172.21.15.6:0/1475957882 >> 172.21.15.6:0/1475957882 conn(0x55a5d57835b0 msgr2=0x55a5d5788ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.253+0000 7f3bcd808b80 1 -- 172.21.15.6:0/1475957882 shutdown_connections 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.254+0000 7f3bcd808b80 1 -- 172.21.15.6:0/1475957882 wait complete. 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.254+0000 7f3bb37fe700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 0x7f3b9c0a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40cb3f0 tx=0x7f3ba4009ca0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.254+0000 7f3bcd808b80 1 -- start start 2023-11-17T00:09:15.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.254+0000 7f3bcd808b80 1 Processor -- start 2023-11-17T00:09:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d56808e0 con 0x55a5d579f6d0 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d5680580 con 0x55a5d57a3ab0 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d5562920 con 0x55a5d579edc0 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bcd808b80 10 client.0 _ll_register_callbacks cb 0x55a5d569e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43998/0 (socket says 172.21.15.6:43998) 2023-11-17T00:09:15.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bb3fff700 1 -- 172.21.15.6:0/995784288 learned_addr learned my addr 172.21.15.6:0/995784288 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:15.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.255+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 -- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d57a3ab0 msgr2=0x55a5d57a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 -- 172.21.15.6:0/995784288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 msgr2=0x55a5d57a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 -- 172.21.15.6:0/995784288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac003b90 con 0x55a5d579edc0 2023-11-17T00:09:15.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007ae0 tx=0x7f3bac009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6c605640a07d5d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:15.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.256+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac007bf0 con 0x55a5d579edc0 2023-11-17T00:09:15.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3bac00e700 con 0x55a5d579edc0 2023-11-17T00:09:15.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5d5780d70 con 0x55a5d579edc0 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bac00e8c0 con 0x55a5d579edc0 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3bcd808b80 10 client.25588 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a5d56a5190 con 0x55a5d579edc0 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac032030 con 0x55a5d579edc0 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.257+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3bac015a30 con 0x55a5d579edc0 2023-11-17T00:09:15.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 10 client.25588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 20 client.25588 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d57b8860 0x55a5d57bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 4 client.25588 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5d56a10a0 con 0x55a5d57b8860 2023-11-17T00:09:15.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.258+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d57b8860 0x55a5d57bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.259+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d57b8860 0x55a5d57bac70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64871b7d3220987e server_cookie=f09f07b16eebd6c3 in_seq=0 out_seq=0 2023-11-17T00:09:15.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.259+0000 7f3b92ffd700 10 client.25588 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:15.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.259+0000 7f3b92ffd700 1 -- 172.21.15.6:0/995784288 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a5d56a10a0 con 0x55a5d57b8860 2023-11-17T00:09:15.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.259+0000 7f3b92ffd700 10 client.25588 handle_command_reply: tid=0 2023-11-17T00:09:15.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.259+0000 7f3bcd808b80 1 client.25588 shutdown 2023-11-17T00:09:15.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 msgr2=0x55a5d57ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007ae0 tx=0x7f3bac009ac0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 shutdown_connections 2023-11-17T00:09:15.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d57a3ab0 0x55a5d57a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57ab250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d57b8860 0x55a5d57bac70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/995784288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3b92ffd700 0 client.25588 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 >> 172.21.15.6:0/995784288 conn(0x55a5d57835b0 msgr2=0x55a5d5788030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 shutdown_connections 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 1 -- 172.21.15.6:0/995784288 wait complete. 2023-11-17T00:09:15.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.260+0000 7f3bcd808b80 20 client.25588 trim_cache size 0 max 16384 2023-11-17T00:09:15.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 Processor -- start 2023-11-17T00:09:15.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 -- start start 2023-11-17T00:09:15.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d57bb3b0 con 0x55a5d57a3ab0 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d5783370 con 0x55a5d579f6d0 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.287+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d57a8160 con 0x55a5d579edc0 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44002/0 (socket says 172.21.15.6:44002) 2023-11-17T00:09:15.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 -- 172.21.15.6:0/986410574 learned_addr learned my addr 172.21.15.6:0/986410574 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:15.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 -- 172.21.15.6:0/986410574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 msgr2=0x55a5d57aad80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:09:15.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57aad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 -- 172.21.15.6:0/986410574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 msgr2=0x55a5d57887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 -- 172.21.15.6:0/986410574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5d56a4ee0 con 0x55a5d579edc0 2023-11-17T00:09:15.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.288+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bac015e10 tx=0x7f3bac009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b68b01efe6323ba server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3b9a7fc700 1 -- 172.21.15.6:0/986410574 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac022070 con 0x55a5d579edc0 2023-11-17T00:09:15.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3b9a7fc700 1 -- 172.21.15.6:0/986410574 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac032070 con 0x55a5d579edc0 2023-11-17T00:09:15.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3b9a7fc700 1 -- 172.21.15.6:0/986410574 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01ea50 con 0x55a5d579edc0 2023-11-17T00:09:15.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 -- 172.21.15.6:0/986410574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 msgr2=0x55a5d57a7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bac015e10 tx=0x7f3bac009ac0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 -- 172.21.15.6:0/986410574 shutdown_connections 2023-11-17T00:09:15.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d57a3ab0 0x55a5d57887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d579edc0 0x55a5d57a7c20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/986410574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579f6d0 0x55a5d57aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.289+0000 7f3bcd808b80 1 -- 172.21.15.6:0/986410574 >> 172.21.15.6:0/986410574 conn(0x55a5d57835b0 msgr2=0x55a5d57a5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.290+0000 7f3bcd808b80 1 -- 172.21.15.6:0/986410574 shutdown_connections 2023-11-17T00:09:15.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.290+0000 7f3bcd808b80 1 -- 172.21.15.6:0/986410574 wait complete. 2023-11-17T00:09:15.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.290+0000 7f3bcd808b80 1 -- start start 2023-11-17T00:09:15.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.290+0000 7f3bcd808b80 1 Processor -- start 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 0x55a5d57abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 0x55a5d580b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d57aca70 con 0x55a5d579f6d0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d580bc10 con 0x55a5d579edc0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d580bdd0 con 0x55a5d57a3ab0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.291+0000 7f3bcd808b80 10 client.0 _ll_register_callbacks cb 0x7f3bac007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.292+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 0x55a5d57abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.292+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 0x55a5d57abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34338/0 (socket says 172.21.15.6:34338) 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.292+0000 7f3bb3fff700 1 -- 172.21.15.6:0/2255342601 learned_addr learned my addr 172.21.15.6:0/2255342601 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.292+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 0x55a5d580b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 -- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 msgr2=0x55a5d57abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 0x55a5d57abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 -- 172.21.15.6:0/2255342601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 msgr2=0x55a5d57ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 -- 172.21.15.6:0/2255342601 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5d56a5190 con 0x55a5d57a3ab0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bb3fff700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 0x55a5d580b6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004f90 tx=0x7f3bac000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6daa8173f3b660c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac022030 con 0x55a5d57a3ab0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bac00f040 con 0x55a5d57a3ab0 2023-11-17T00:09:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.293+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3bac01fa30 con 0x55a5d57a3ab0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bac00e700 con 0x55a5d57a3ab0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 10 client.25594 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a5d54f5c60 con 0x55a5d57a3ab0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac00f040 con 0x55a5d57a3ab0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3bac015890 con 0x55a5d57a3ab0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 20 client.25594 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d580d050 0x55a5d580f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 4 client.25594 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.294+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a5d56ad660 con 0x55a5d580d050 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d580d050 0x55a5d580f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3bb2ffd700 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d580d050 0x55a5d580f430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a0e9ca7029185 server_cookie=699e0c30e7ee4dbf in_seq=0 out_seq=0 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3b92ffd700 10 client.25594 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3b92ffd700 1 -- 172.21.15.6:0/2255342601 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a5d56ad660 con 0x55a5d580d050 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3b92ffd700 10 client.25594 handle_command_reply: tid=0 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.295+0000 7f3bcd808b80 1 client.25594 shutdown 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 msgr2=0x55a5d580b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 0x55a5d580b6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004f90 tx=0x7f3bac000f40 comp rx=0 tx=0).stop 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 shutdown_connections 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a5d57a3ab0 0x55a5d580b6d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a5d579edc0 0x55a5d57abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a5d580d050 0x55a5d580f430 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 --2- 172.21.15.6:0/2255342601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a5d579f6d0 0x55a5d57ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 >> 172.21.15.6:0/2255342601 conn(0x55a5d57835b0 msgr2=0x55a5d57a5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3b92ffd700 0 client.25594 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 shutdown_connections 2023-11-17T00:09:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 1 -- 172.21.15.6:0/2255342601 wait complete. 2023-11-17T00:09:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.296+0000 7f3bcd808b80 20 client.25594 trim_cache size 0 max 16384 2023-11-17T00:09:15.532 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 msgr2=0x7f3b9c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 0x7f3b9c0a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40cb3f0 tx=0x7f3ba4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 msgr2=0x7f3bb40ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb40ff160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b2e0 tx=0x7f3ba4004730 comp rx=0 tx=0).stop 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 shutdown_connections 2023-11-17T00:09:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3b9c0a1460 0x7f3b9c0a3910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb4106080 0x7f3bb40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bb4067dc0 0x7f3bb40ff160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 --2- 172.21.15.6:0/21697692 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3bb4068750 0x7f3bb40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 >> 172.21.15.6:0/21697692 conn(0x7f3bb4074ef0 msgr2=0x7f3bb40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 shutdown_connections 2023-11-17T00:09:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:15.297+0000 7f3bb8b9d700 1 -- 172.21.15.6:0/21697692 wait complete. 2023-11-17T00:09:15.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2430344444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:15.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2430344444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:15.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2430344444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:16.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:09:16.538 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:09:16.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:16 smithi006 ceph-mon[97840]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-11-17T00:09:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:16 smithi087 ceph-mon[131182]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-11-17T00:09:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:16 smithi186 ceph-mon[131487]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 -- 172.21.15.6:0/1711240471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08067dc0 msgr2=0x7f3f08068210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 --2- 172.21.15.6:0/1711240471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08067dc0 0x7f3f08068210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009990 tx=0x7f3ef8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 -- 172.21.15.6:0/1711240471 shutdown_connections 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 --2- 172.21.15.6:0/1711240471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08068750 0x7f3f0810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 --2- 172.21.15.6:0/1711240471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08106080 0x7f3f08106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 --2- 172.21.15.6:0/1711240471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08067dc0 0x7f3f08068210 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 -- 172.21.15.6:0/1711240471 >> 172.21.15.6:0/1711240471 conn(0x7f3f08074ef0 msgr2=0x7f3f080752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 -- 172.21.15.6:0/1711240471 shutdown_connections 2023-11-17T00:09:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.895+0000 7f3f0759e700 1 -- 172.21.15.6:0/1711240471 wait complete. 2023-11-17T00:09:16.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.897+0000 7f3f0759e700 1 Processor -- start 2023-11-17T00:09:16.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.897+0000 7f3f0759e700 1 -- start start 2023-11-17T00:09:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 0x7f3f081961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08068750 0x7f3f08196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0819b1b0 con 0x7f3f08068750 2023-11-17T00:09:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0819b320 con 0x7f3f08067dc0 2023-11-17T00:09:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0759e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0819b4e0 con 0x7f3f08106080 2023-11-17T00:09:16.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 0x7f3f081961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f06d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f06d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44020/0 (socket says 172.21.15.6:44020) 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f06d9d700 1 -- 172.21.15.6:0/3335468643 learned_addr learned my addr 172.21.15.6:0/3335468643 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08068750 0x7f3f08196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.898+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 0x7f3f081961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34356/0 (socket says 172.21.15.6:34356) 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 -- 172.21.15.6:0/3335468643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 msgr2=0x7f3f081961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 0x7f3f081961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 -- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08068750 msgr2=0x7f3f08196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08068750 0x7f3f08196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 -- 172.21.15.6:0/3335468643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0009520 con 0x7f3f08106080 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.899+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000ef30 tx=0x7f3f0000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.900+0000 7f3eff7fe700 1 -- 172.21.15.6:0/3335468643 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00004bb0 con 0x7f3f08106080 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.900+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef80095f0 con 0x7f3f08106080 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.900+0000 7f3eff7fe700 1 -- 172.21.15.6:0/3335468643 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0000f040 con 0x7f3f08106080 2023-11-17T00:09:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.900+0000 7f3eff7fe700 1 -- 172.21.15.6:0/3335468643 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00004020 con 0x7f3f08106080 2023-11-17T00:09:16.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.900+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f08110130 con 0x7f3f08106080 2023-11-17T00:09:16.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.905+0000 7f3eff7fe700 1 -- 172.21.15.6:0/3335468643 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3f00003a00 con 0x7f3f08106080 2023-11-17T00:09:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.908+0000 7f3f206f6b80 1 Processor -- start 2023-11-17T00:09:16.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.908+0000 7f3eff7fe700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 0x7f3ef40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.908+0000 7f3eff7fe700 1 -- 172.21.15.6:0/3335468643 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3f000d09c0 con 0x7f3f08106080 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.908+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 0x7f3ef40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 -- start start 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec5ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec5e300 0x55e8dec5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8deb560e0 con 0x55e8dec59f50 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8deba8920 con 0x55e8dec5e300 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.909+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8deba1db0 con 0x55e8dec59640 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f05d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec5ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44030/0 (socket says 172.21.15.6:44030) 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f0659c700 1 -- 172.21.15.6:0/1793717258 learned_addr learned my addr 172.21.15.6:0/1793717258 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 0x7f3ef40a3960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3f081977e0 tx=0x7f3ef8005ad0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.910+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec5e300 0x55e8dec5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 -- 172.21.15.6:0/1793717258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 msgr2=0x55e8dec59a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 -- 172.21.15.6:0/1793717258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec5e300 msgr2=0x55e8dec5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec5e300 0x55e8dec5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 -- 172.21.15.6:0/1793717258 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8dea236b0 con 0x55e8dec59f50 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f0659c700 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.911+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec5ddc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000c00 tx=0x7f3ef8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9676ef305af369c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.912+0000 7f3efcff9700 1 -- 172.21.15.6:0/1793717258 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8004700 con 0x55e8dec59f50 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.912+0000 7f3efcff9700 1 -- 172.21.15.6:0/1793717258 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef801e040 con 0x55e8dec59f50 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.912+0000 7f3efcff9700 1 -- 172.21.15.6:0/1793717258 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800f930 con 0x55e8dec59f50 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 -- 172.21.15.6:0/1793717258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 msgr2=0x55e8dec5ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec5ddc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000c00 tx=0x7f3ef8020000 comp rx=0 tx=0).stop 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 -- 172.21.15.6:0/1793717258 shutdown_connections 2023-11-17T00:09:16.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec5e300 0x55e8dec5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59640 0x55e8dec59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/1793717258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec5ddc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 -- 172.21.15.6:0/1793717258 >> 172.21.15.6:0/1793717258 conn(0x55e8dec46630 msgr2=0x55e8dec4bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.913+0000 7f3f206f6b80 1 -- 172.21.15.6:0/1793717258 shutdown_connections 2023-11-17T00:09:16.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.914+0000 7f3f206f6b80 1 -- 172.21.15.6:0/1793717258 wait complete. 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.914+0000 7f3f206f6b80 1 -- start start 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 Processor -- start 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec5e300 0x55e8dec63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8deb43960 con 0x55e8dec5e300 2023-11-17T00:09:16.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8deb43600 con 0x55e8dec59640 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dea259a0 con 0x55e8dec59f50 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.915+0000 7f3f206f6b80 10 client.0 _ll_register_callbacks cb 0x55e8deb61800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44036/0 (socket says 172.21.15.6:44036) 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 -- 172.21.15.6:0/743508234 learned_addr learned my addr 172.21.15.6:0/743508234 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:16.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec5e300 0x55e8dec63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 -- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 msgr2=0x55e8dec65af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 -- 172.21.15.6:0/743508234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec5e300 msgr2=0x55e8dec63440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec5e300 0x55e8dec63440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.916+0000 7f3f05d9b700 1 -- 172.21.15.6:0/743508234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0810d720 con 0x55e8dec59f50 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.917+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ce0 tx=0x7f3ef800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73af94f9c9febd2d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.917+0000 7f3eedffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.917+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8034070 con 0x55e8dec59f50 2023-11-17T00:09:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.917+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8dec43df0 con 0x55e8dec59f50 2023-11-17T00:09:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.917+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ef8004710 con 0x55e8dec59f50 2023-11-17T00:09:16.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.918+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef80048d0 con 0x55e8dec59f50 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.919+0000 7f3f206f6b80 10 client.25606 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.919+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e8deb68210 con 0x55e8dec59f50 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8dec43df0 con 0x55e8dec59f50 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ef8030750 con 0x55e8dec59f50 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:16.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 20 client.25606 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec72f90 0x55e8dec753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 4 client.25606 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.920+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8deb64120 con 0x55e8dec72f90 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.921+0000 7f3f0659c700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec72f90 0x55e8dec753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.922+0000 7f3f0659c700 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec72f90 0x55e8dec753a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed80b569840d32c0 server_cookie=bf185b0ab3a3c363 in_seq=0 out_seq=0 2023-11-17T00:09:16.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.922+0000 7f3eedffb700 10 client.25606 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:16.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.922+0000 7f3eedffb700 1 -- 172.21.15.6:0/743508234 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8deb64120 con 0x55e8dec72f90 2023-11-17T00:09:16.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3eedffb700 10 client.25606 handle_command_reply: tid=0 2023-11-17T00:09:16.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 client.25606 shutdown 2023-11-17T00:09:16.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 msgr2=0x55e8dec62d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ce0 tx=0x7f3ef800f740 comp rx=0 tx=0).stop 2023-11-17T00:09:16.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 shutdown_connections 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec72f90 0x55e8dec753a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec5e300 0x55e8dec63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3eedffb700 0 client.25606 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/743508234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec59f50 0x55e8dec62d00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 >> 172.21.15.6:0/743508234 conn(0x55e8dec46630 msgr2=0x55e8dec4b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 shutdown_connections 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.923+0000 7f3f206f6b80 1 -- 172.21.15.6:0/743508234 wait complete. 2023-11-17T00:09:16.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.924+0000 7f3f206f6b80 20 client.25606 trim_cache size 0 max 16384 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 Processor -- start 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 -- start start 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dec46160 con 0x55e8dec59f50 2023-11-17T00:09:16.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dec43930 con 0x55e8dec59640 2023-11-17T00:09:16.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.959+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8dec75a00 con 0x55e8dec46630 2023-11-17T00:09:16.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44046/0 (socket says 172.21.15.6:44046) 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 -- 172.21.15.6:0/2204339419 learned_addr learned my addr 172.21.15.6:0/2204339419 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 -- 172.21.15.6:0/2204339419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 msgr2=0x55e8dec65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 -- 172.21.15.6:0/2204339419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 msgr2=0x55e8dec4b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec4b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.960+0000 7f3f0659c700 1 -- 172.21.15.6:0/2204339419 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8deb67f60 con 0x55e8dec46630 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.961+0000 7f3f0659c700 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55e8dec75c70 tx=0x7f3ef0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a64bee30f07978 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.961+0000 7f3efd7fa700 1 -- 172.21.15.6:0/2204339419 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0016030 con 0x55e8dec46630 2023-11-17T00:09:16.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.961+0000 7f3efd7fa700 1 -- 172.21.15.6:0/2204339419 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef0017040 con 0x55e8dec46630 2023-11-17T00:09:16.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.961+0000 7f3efd7fa700 1 -- 172.21.15.6:0/2204339419 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0003e80 con 0x55e8dec46630 2023-11-17T00:09:16.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 -- 172.21.15.6:0/2204339419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 msgr2=0x55e8dec5da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55e8dec75c70 tx=0x7f3ef0009260 comp rx=0 tx=0).stop 2023-11-17T00:09:16.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 -- 172.21.15.6:0/2204339419 shutdown_connections 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59f50 0x55e8dec4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec5da20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/2204339419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59640 0x55e8dec65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 -- 172.21.15.6:0/2204339419 >> 172.21.15.6:0/2204339419 conn(0x55e8dec6fa00 msgr2=0x55e8dec5d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 -- 172.21.15.6:0/2204339419 shutdown_connections 2023-11-17T00:09:16.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.962+0000 7f3f206f6b80 1 -- 172.21.15.6:0/2204339419 wait complete. 2023-11-17T00:09:16.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.963+0000 7f3f206f6b80 1 -- start start 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.963+0000 7f3f206f6b80 1 Processor -- start 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59640 0x55e8dec69860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59f50 0x55e8decc0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8decc13a0 con 0x55e8dec59640 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8decc1510 con 0x55e8dec59f50 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8decc1680 con 0x55e8dec46630 2023-11-17T00:09:16.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f206f6b80 10 client.0 _ll_register_callbacks cb 0x7f3f000991c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.964+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44060/0 (socket says 172.21.15.6:44060) 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 -- 172.21.15.6:0/3551704700 learned_addr learned my addr 172.21.15.6:0/3551704700 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f06d9d700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59f50 0x55e8decc0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59640 0x55e8dec69860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 -- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59f50 msgr2=0x55e8decc0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59f50 0x55e8decc0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 -- 172.21.15.6:0/3551704700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59640 msgr2=0x55e8dec69860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59640 0x55e8dec69860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 -- 172.21.15.6:0/3551704700 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0009e70 con 0x55e8dec46630 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3f0659c700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0025040 tx=0x7f3ef001f290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5e506ee26f7d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:16.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.965+0000 7f3eedffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0016040 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ef0005040 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ef801e040 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ef0005200 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3f206f6b80 10 client.25612 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.966+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e8de9b8ce0 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef801e040 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ef0021d90 con 0x55e8dec46630 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 20 client.25612 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec5e300 0x55e8deccad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:16.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 4 client.25612 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:09:16.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e8deb706e0 con 0x55e8dec5e300 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.967+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec5e300 0x55e8deccad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.968+0000 7f3f05d9b700 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec5e300 0x55e8deccad40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4b04cf4726f6b9 server_cookie=60f0bedf3744f524 in_seq=0 out_seq=0 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.968+0000 7f3eedffb700 10 client.25612 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.969+0000 7f3eedffb700 1 -- 172.21.15.6:0/3551704700 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e8deb706e0 con 0x55e8dec5e300 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.969+0000 7f3eedffb700 10 client.25612 handle_command_reply: tid=0 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.969+0000 7f3f206f6b80 1 client.25612 shutdown 2023-11-17T00:09:16.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 msgr2=0x55e8dec69320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0025040 tx=0x7f3ef001f290 comp rx=0 tx=0).stop 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 shutdown_connections 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e8dec59f50 0x55e8decc0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8dec46630 0x55e8dec69320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e8dec5e300 0x55e8deccad40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 --2- 172.21.15.6:0/3551704700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e8dec59640 0x55e8dec69860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 >> 172.21.15.6:0/3551704700 conn(0x55e8dec6fa00 msgr2=0x55e8dec4fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3eedffb700 0 client.25612 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.970+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 shutdown_connections 2023-11-17T00:09:16.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.971+0000 7f3f206f6b80 1 -- 172.21.15.6:0/3551704700 wait complete. 2023-11-17T00:09:16.975 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:09:16.975 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:09:16.975 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "ece5ff68-0757-46d9-8505-4c7e046a6005", 2023-11-17T00:09:16.975 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:09:16.975 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:09:16.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.971+0000 7f3f206f6b80 20 client.25612 trim_cache size 0 max 16384 2023-11-17T00:09:16.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 msgr2=0x7f3ef40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 0x7f3ef40a3960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3f081977e0 tx=0x7f3ef8005ad0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 msgr2=0x7f3f0819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000ef30 tx=0x7f3f0000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 shutdown_connections 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3ef40a14b0 0x7f3ef40a3960 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f08106080 0x7f3f0819ab70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08067dc0 0x7f3f081961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 --2- 172.21.15.6:0/3335468643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f08068750 0x7f3f08196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 >> 172.21.15.6:0/3335468643 conn(0x7f3f08074ef0 msgr2=0x7f3f080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 shutdown_connections 2023-11-17T00:09:16.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:16.972+0000 7f3f0759e700 1 -- 172.21.15.6:0/3335468643 wait complete. 2023-11-17T00:09:17.002 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 -- 172.21.15.6:0/4173147583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978100c50 msgr2=0x7f5978101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 --2- 172.21.15.6:0/4173147583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978100c50 0x7f5978101020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f596800faf0 tx=0x7f596800fe00 comp rx=0 tx=0).stop 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 -- 172.21.15.6:0/4173147583 shutdown_connections 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 --2- 172.21.15.6:0/4173147583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5978105560 0x7f597810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 --2- 172.21.15.6:0/4173147583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978100c50 0x7f5978101020 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 --2- 172.21.15.6:0/4173147583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59781015f0 0x7f5978104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 -- 172.21.15.6:0/4173147583 >> 172.21.15.6:0/4173147583 conn(0x7f5978077500 msgr2=0x7f5978077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.234+0000 7f597cfff700 1 -- 172.21.15.6:0/4173147583 shutdown_connections 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.235+0000 7f597cfff700 1 -- 172.21.15.6:0/4173147583 wait complete. 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.236+0000 7f597cfff700 1 Processor -- start 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.236+0000 7f597cfff700 1 -- start start 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 0x7f59781a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978105560 0x7f597819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819cea0 con 0x7f59781015f0 2023-11-17T00:09:17.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819d010 con 0x7f5978100c50 2023-11-17T00:09:17.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819d1d0 con 0x7f5978105560 2023-11-17T00:09:17.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58506/0 (socket says 172.21.15.6:58506) 2023-11-17T00:09:17.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f5975d9b700 1 -- 172.21.15.6:0/4145618437 learned_addr learned my addr 172.21.15.6:0/4145618437 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f597659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 0x7f59781a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.237+0000 7f5976d9d700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978105560 0x7f597819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 -- 172.21.15.6:0/4145618437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978105560 msgr2=0x7f597819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978105560 0x7f597819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 -- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 msgr2=0x7f59781a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 -- 172.21.15.6:0/4145618437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596800f750 con 0x7f5978100c50 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f5975d9b700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.238+0000 7f597659c700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 0x7f59781a27c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f596800fa40 tx=0x7f5968005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.239+0000 7f59677fe700 1 -- 172.21.15.6:0/4145618437 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968017830 con 0x7f5978100c50 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.239+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781169d0 con 0x7f5978100c50 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.239+0000 7f59677fe700 1 -- 172.21.15.6:0/4145618437 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f596800f4c0 con 0x7f5978100c50 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.239+0000 7f59677fe700 1 -- 172.21.15.6:0/4145618437 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596801fdb0 con 0x7f5978100c50 2023-11-17T00:09:17.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.239+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978116840 con 0x7f5978100c50 2023-11-17T00:09:17.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.244+0000 7f59677fe700 1 -- 172.21.15.6:0/4145618437 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f596801f440 con 0x7f5978100c50 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.246+0000 7f59677fe700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 0x7f59600a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.246+0000 7f59909f5b80 1 Processor -- start 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.246+0000 7f59677fe700 1 -- 172.21.15.6:0/4145618437 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f59680cead0 con 0x7f5978100c50 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.246+0000 7f5975d9b700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 0x7f59600a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 -- start start 2023-11-17T00:09:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f1487f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f5975d9b700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 0x7f59600a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59781052d0 tx=0x7f596c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f148cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f138d060 con 0x5625f1488470 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f13df8a0 con 0x5625f148c8b0 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f59909f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f13d8d30 con 0x5625f1487b60 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.247+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58516/0 (socket says 172.21.15.6:58516) 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f597659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f1487f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 -- 172.21.15.6:0/2790042583 learned_addr learned my addr 172.21.15.6:0/2790042583 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5976d9d700 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f148cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 -- 172.21.15.6:0/2790042583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 msgr2=0x5625f1487f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f1487f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 -- 172.21.15.6:0/2790042583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 msgr2=0x5625f148cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f148cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 -- 172.21.15.6:0/2790042583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f125a630 con 0x5625f1488470 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.248+0000 7f5975d9b700 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f596c00e6e0 tx=0x7f596c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a396991559f09a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.249+0000 7f5964ff9700 1 -- 172.21.15.6:0/2790042583 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c026630 con 0x5625f1488470 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.249+0000 7f5964ff9700 1 -- 172.21.15.6:0/2790042583 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f596c021070 con 0x5625f1488470 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.249+0000 7f5964ff9700 1 -- 172.21.15.6:0/2790042583 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c01d690 con 0x5625f1488470 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.249+0000 7f59909f5b80 1 -- 172.21.15.6:0/2790042583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 msgr2=0x5625f148c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.249+0000 7f59909f5b80 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f596c00e6e0 tx=0x7f596c014000 comp rx=0 tx=0).stop 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 -- 172.21.15.6:0/2790042583 shutdown_connections 2023-11-17T00:09:17.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f148cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f1487f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 --2- 172.21.15.6:0/2790042583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f148c370 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 -- 172.21.15.6:0/2790042583 >> 172.21.15.6:0/2790042583 conn(0x5625f147d360 msgr2=0x5625f1482b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 -- 172.21.15.6:0/2790042583 shutdown_connections 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 -- 172.21.15.6:0/2790042583 wait complete. 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.250+0000 7f59909f5b80 1 -- start start 2023-11-17T00:09:17.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 Processor -- start 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1491340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f1491dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f137a8e0 con 0x5625f148c8b0 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f137a580 con 0x5625f1487b60 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f125c920 con 0x5625f1488470 2023-11-17T00:09:17.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f59909f5b80 10 client.0 _ll_register_callbacks cb 0x5625f1398780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.251+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44096/0 (socket says 172.21.15.6:44096) 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 -- 172.21.15.6:0/3288866715 learned_addr learned my addr 172.21.15.6:0/3288866715 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f597659c700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1491340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 -- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 msgr2=0x5625f1491340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5976d9d700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f1491dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1491340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 -- 172.21.15.6:0/3288866715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 msgr2=0x5625f1491dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f1491dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5975d9b700 1 -- 172.21.15.6:0/3288866715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c017070 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.252+0000 7f5976d9d700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f1491dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f5975d9b700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f596c01ce70 tx=0x7f596c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be74317982ea4a2b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f595dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c01c860 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625f147ab20 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f596c01c9c0 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f596c005040 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f59909f5b80 10 client.25624 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.253+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5625f139f190 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625f147ab20 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f596c004ce0 con 0x5625f1488470 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 10 client.25624 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:17.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:17.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 20 client.25624 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:17.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f14a1830 0x5625f14a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 4 client.25624 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:17.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.254+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625f139b0a0 con 0x5625f14a1830 2023-11-17T00:09:17.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.255+0000 7f597659c700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f14a1830 0x5625f14a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.255+0000 7f597659c700 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f14a1830 0x5625f14a3c40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30ea1895ee7252b server_cookie=38c379666fa148a7 in_seq=0 out_seq=0 2023-11-17T00:09:17.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.255+0000 7f595dffb700 10 client.25624 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:17.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f595dffb700 1 -- 172.21.15.6:0/3288866715 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5625f139b0a0 con 0x5625f14a1830 2023-11-17T00:09:17.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f595dffb700 10 client.25624 handle_command_reply: tid=0 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 client.25624 shutdown 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 msgr2=0x5625f1491880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f596c01ce70 tx=0x7f596c00ee80 comp rx=0 tx=0).stop 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 shutdown_connections 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f14a1830 0x5625f14a3c40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f1491dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1491340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f595dffb700 0 client.25624 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3288866715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1491880 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 >> 172.21.15.6:0/3288866715 conn(0x5625f147d360 msgr2=0x5625f148dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 shutdown_connections 2023-11-17T00:09:17.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.256+0000 7f59909f5b80 1 -- 172.21.15.6:0/3288866715 wait complete. 2023-11-17T00:09:17.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.257+0000 7f59909f5b80 20 client.25624 trim_cache size 0 max 16384 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.274+0000 7f59909f5b80 1 Processor -- start 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.274+0000 7f59909f5b80 1 -- start start 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f148a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f14989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f1491c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f149e300 con 0x5625f1488470 2023-11-17T00:09:17.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f14841c0 con 0x5625f148c8b0 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f59909f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f148b520 con 0x5625f1487b60 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f597659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f148a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f597659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f148a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44100/0 (socket says 172.21.15.6:44100) 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f597659c700 1 -- 172.21.15.6:0/1095250353 learned_addr learned my addr 172.21.15.6:0/1095250353 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f5976d9d700 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f1491c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.275+0000 7f5975d9b700 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f14989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 -- 172.21.15.6:0/1095250353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 msgr2=0x5625f148a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f148a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 -- 172.21.15.6:0/1095250353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 msgr2=0x5625f14989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f14989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 -- 172.21.15.6:0/1095250353 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f147eb00 con 0x5625f148c8b0 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f5976d9d700 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f1491c40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59781a34c0 tx=0x7f5970009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9109f08382f530d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f59657fa700 1 -- 172.21.15.6:0/1095250353 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970004bb0 con 0x5625f148c8b0 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f59657fa700 1 -- 172.21.15.6:0/1095250353 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f597000f040 con 0x5625f148c8b0 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.276+0000 7f59657fa700 1 -- 172.21.15.6:0/1095250353 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970016d10 con 0x5625f148c8b0 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 -- 172.21.15.6:0/1095250353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 msgr2=0x5625f1491c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f1491c40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59781a34c0 tx=0x7f5970009d30 comp rx=0 tx=0).stop 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 -- 172.21.15.6:0/1095250353 shutdown_connections 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f148c8b0 0x5625f1491c40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1487b60 0x5625f148a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 --2- 172.21.15.6:0/1095250353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f1488470 0x5625f14989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 -- 172.21.15.6:0/1095250353 >> 172.21.15.6:0/1095250353 conn(0x5625f147d360 msgr2=0x5625f148aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 -- 172.21.15.6:0/1095250353 shutdown_connections 2023-11-17T00:09:17.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.277+0000 7f59909f5b80 1 -- 172.21.15.6:0/1095250353 wait complete. 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.278+0000 7f59909f5b80 1 -- start start 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.278+0000 7f59909f5b80 1 Processor -- start 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1498120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1498660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f14964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f1496b30 con 0x5625f148c8b0 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f1496ca0 con 0x5625f1487b60 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f1496e60 con 0x5625f1488470 2023-11-17T00:09:17.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f59909f5b80 10 client.0 _ll_register_callbacks cb 0x7f5970009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f597659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1498120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f5976d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f14964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f597659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1498120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34432/0 (socket says 172.21.15.6:34432) 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.279+0000 7f597659c700 1 -- 172.21.15.6:0/3379176377 learned_addr learned my addr 172.21.15.6:0/3379176377 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 -- 172.21.15.6:0/3379176377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 msgr2=0x5625f1498660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1498660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 -- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 msgr2=0x5625f1498120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1498120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 -- 172.21.15.6:0/3379176377 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5970009a10 con 0x5625f148c8b0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f5976d9d700 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f14964e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5970009710 tx=0x7f5970016710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a99a710b18a1b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f595dffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.280+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970007d70 con 0x5625f148c8b0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f596c021070 con 0x5625f148c8b0 2023-11-17T00:09:17.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5970007b80 con 0x5625f148c8b0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5970003c10 con 0x5625f148c8b0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 10 client.15576 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5625f1206de0 con 0x5625f148c8b0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f596c021070 con 0x5625f148c8b0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f59700045f0 con 0x5625f148c8b0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 20 client.15576 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f15076c0 0x5625f1509ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 4 client.15576 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.281+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625f13a7660 con 0x5625f15076c0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.282+0000 7f597659c700 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f15076c0 0x5625f1509ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:17.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.282+0000 7f597659c700 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f15076c0 0x5625f1509ad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d365b3342631d99 server_cookie=a81a07be697f7230 in_seq=0 out_seq=0 2023-11-17T00:09:17.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.282+0000 7f595dffb700 10 client.15576 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f595dffb700 1 -- 172.21.15.6:0/3379176377 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5625f13a7660 con 0x5625f15076c0 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f595dffb700 10 client.15576 handle_command_reply: tid=0 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 client.15576 shutdown 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 msgr2=0x5625f14964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f14964e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5970009710 tx=0x7f5970016710 comp rx=0 tx=0).stop 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 shutdown_connections 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5625f15076c0 0x5625f1509ad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5625f148c8b0 0x5625f14964e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5625f1487b60 0x5625f1498120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 --2- 172.21.15.6:0/3379176377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5625f1488470 0x5625f1498660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 >> 172.21.15.6:0/3379176377 conn(0x5625f147d360 msgr2=0x5625f1491900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f595dffb700 0 client.15576 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 shutdown_connections 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.283+0000 7f59909f5b80 1 -- 172.21.15.6:0/3379176377 wait complete. 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f59909f5b80 20 client.15576 trim_cache size 0 max 16384 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:09:17.289 INFO:teuthology.orchestra.run.smithi006.stdout: "ece5ff68-0757-46d9-8505-4c7e046a6005": { 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "ece5ff68-0757-46d9-8505-4c7e046a6005", 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:09:17.290 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 msgr2=0x7f59600a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 0x7f59600a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59781052d0 tx=0x7f596c009260 comp rx=0 tx=0).stop 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 msgr2=0x7f59781a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 0x7f59781a27c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f596800fa40 tx=0x7f5968005b00 comp rx=0 tx=0).stop 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 shutdown_connections 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59600a13e0 0x7f59600a3890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5978105560 0x7f597819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5978100c50 0x7f59781a27c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 --2- 172.21.15.6:0/4145618437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59781015f0 0x7f59781a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.284+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 >> 172.21.15.6:0/4145618437 conn(0x7f5978077500 msgr2=0x7f59780feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.285+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 shutdown_connections 2023-11-17T00:09:17.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:17.285+0000 7f597cfff700 1 -- 172.21.15.6:0/4145618437 wait complete. 2023-11-17T00:09:17.305 INFO:tasks.cephfs.filesystem:scrub status for tag:ece5ff68-0757-46d9-8505-4c7e046a6005 - {'path': '/', 'tag': 'ece5ff68-0757-46d9-8505-4c7e046a6005', 'options': 'recursive,force'} 2023-11-17T00:09:18.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:18 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:09:18.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:18 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:18 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:09:18.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:18 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:18 smithi006 ceph-mon[97840]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-17T00:09:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:18 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:09:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:18 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:18 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:09:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:18 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:18 smithi087 ceph-mon[131182]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-17T00:09:18.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:18 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:09:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:18 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:18 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:09:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:18 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:09:18.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:18 smithi186 ceph-mon[131487]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-17T00:09:20.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:20 smithi006 ceph-mon[97840]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-11-17T00:09:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:20 smithi087 ceph-mon[131182]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-11-17T00:09:20.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:20 smithi186 ceph-mon[131487]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-11-17T00:09:21.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:21 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:09:21.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:21 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:09:21.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:21 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:09:22.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:22 smithi006 ceph-mon[97840]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 -- 172.21.15.6:0/1483341786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01701053a0 msgr2=0x7f017010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 --2- 172.21.15.6:0/1483341786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01701053a0 0x7f017010cf70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01680099c0 tx=0x7f0168009cd0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 -- 172.21.15.6:0/1483341786 shutdown_connections 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 --2- 172.21.15.6:0/1483341786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01701053a0 0x7f017010cf70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 --2- 172.21.15.6:0/1483341786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0170100360 0x7f0170100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 --2- 172.21.15.6:0/1483341786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 0x7f0170104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 -- 172.21.15.6:0/1483341786 >> 172.21.15.6:0/1483341786 conn(0x7f01700fbbd0 msgr2=0x7f01700fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 -- 172.21.15.6:0/1483341786 shutdown_connections 2023-11-17T00:09:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.736+0000 7f016f59e700 1 -- 172.21.15.6:0/1483341786 wait complete. 2023-11-17T00:09:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 Processor -- start 2023-11-17T00:09:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 -- start start 2023-11-17T00:09:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 0x7f0170198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 0x7f01701988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 0x7f017019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017019d2f0 con 0x7f0170100d00 2023-11-17T00:09:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017019d460 con 0x7f01701053a0 2023-11-17T00:09:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.738+0000 7f016f59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017019d620 con 0x7f0170100360 2023-11-17T00:09:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 0x7f0170198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 0x7f017019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 0x7f0170198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45070/0 (socket says 172.21.15.6:45070) 2023-11-17T00:09:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016e59c700 1 -- 172.21.15.6:0/3440814480 learned_addr learned my addr 172.21.15.6:0/3440814480 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016dd9b700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 0x7f01701988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 -- 172.21.15.6:0/3440814480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 msgr2=0x7f0170198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 0x7f0170198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 -- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 msgr2=0x7f01701988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 0x7f01701988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.739+0000 7f016ed9d700 1 -- 172.21.15.6:0/3440814480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01680095f0 con 0x7f01701053a0 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f016ed9d700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 0x7f017019ccb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f016800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f015f7fe700 1 -- 172.21.15.6:0/3440814480 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016801d070 con 0x7f01701053a0 2023-11-17T00:09:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01701101c0 con 0x7f01701053a0 2023-11-17T00:09:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f015f7fe700 1 -- 172.21.15.6:0/3440814480 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016800fe90 con 0x7f01701053a0 2023-11-17T00:09:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f015f7fe700 1 -- 172.21.15.6:0/3440814480 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168022e00 con 0x7f01701053a0 2023-11-17T00:09:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.740+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0170110030 con 0x7f01701053a0 2023-11-17T00:09:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.746+0000 7f015f7fe700 1 -- 172.21.15.6:0/3440814480 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f016800f9d0 con 0x7f01701053a0 2023-11-17T00:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.747+0000 7f01882c8b80 1 Processor -- start 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 -- start start 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc05b000 con 0x5573cc155cf0 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc0ad840 con 0x5573cbfb16b0 2023-11-17T00:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.748+0000 7f01882c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc0a6cd0 con 0x5573cc15b990 2023-11-17T00:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45086/0 (socket says 172.21.15.6:45086) 2023-11-17T00:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 -- 172.21.15.6:0/4214491850 learned_addr learned my addr 172.21.15.6:0/4214491850 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:45718/0 (socket says 172.21.15.6:45718) 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f015f7fe700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 0x7f01580a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f015f7fe700 1 -- 172.21.15.6:0/3440814480 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f01680ce0b0 con 0x7f01701053a0 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 -- 172.21.15.6:0/4214491850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 msgr2=0x5573cc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016e59c700 1 -- 172.21.15.6:0/4214491850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 msgr2=0x5573cc156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.749+0000 7f016ed9d700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 0x7f01580a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f016e59c700 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f016e59c700 1 -- 172.21.15.6:0/4214491850 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573cbf285d0 con 0x5573cc15b990 2023-11-17T00:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f016e59c700 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f016000e660 tx=0x7f0160009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ba10b6ec598e6e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f016ed9d700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 0x7f01580a3910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01680cb390 tx=0x7f01680ce370 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f015cff9700 1 -- 172.21.15.6:0/4214491850 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000e040 con 0x5573cc15b990 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f015cff9700 1 -- 172.21.15.6:0/4214491850 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016000f040 con 0x5573cc15b990 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.750+0000 7f015cff9700 1 -- 172.21.15.6:0/4214491850 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160007500 con 0x5573cc15b990 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 -- 172.21.15.6:0/4214491850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 msgr2=0x5573cc1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f016000e660 tx=0x7f0160009920 comp rx=0 tx=0).stop 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 -- 172.21.15.6:0/4214491850 shutdown_connections 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4214491850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc1557b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 -- 172.21.15.6:0/4214491850 >> 172.21.15.6:0/4214491850 conn(0x5573cc14b1a0 msgr2=0x5573cc1503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.751+0000 7f01882c8b80 1 -- 172.21.15.6:0/4214491850 shutdown_connections 2023-11-17T00:09:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.752+0000 7f01882c8b80 1 -- 172.21.15.6:0/4214491850 wait complete. 2023-11-17T00:09:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.752+0000 7f01882c8b80 1 -- start start 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 Processor -- start 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cbfb16b0 0x5573cc161c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc15f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc048880 con 0x5573cc15b990 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc048520 con 0x5573cc155cf0 2023-11-17T00:09:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cbf2a8c0 con 0x5573cbfb16b0 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.753+0000 7f01882c8b80 10 client.0 _ll_register_callbacks cb 0x5573cc066720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:45728/0 (socket says 172.21.15.6:45728) 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 -- 172.21.15.6:0/2496984625 learned_addr learned my addr 172.21.15.6:0/2496984625 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016dd9b700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 -- 172.21.15.6:0/2496984625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cbfb16b0 msgr2=0x5573cc161c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cbfb16b0 0x5573cc161c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 -- 172.21.15.6:0/2496984625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 msgr2=0x5573cc15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.754+0000 7f016e59c700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f016e59c700 1 -- 172.21.15.6:0/2496984625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016000e340 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f016dd9b700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f016e59c700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0160009510 tx=0x7f0160004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1434c347572e5f8c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f0155ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000e020 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0160003a00 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.755+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573cc148960 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.756+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0160004e50 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.756+0000 7f01882c8b80 10 client.25652 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.756+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5573cc06d130 con 0x5573cc155cf0 2023-11-17T00:09:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573cc148960 con 0x5573cc155cf0 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0160015450 con 0x5573cc155cf0 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 20 client.25652 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc166070 0x5573cc168480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 4 client.25652 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.757+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573cc069040 con 0x5573cc166070 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.758+0000 7f016dd9b700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc166070 0x5573cc168480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.758+0000 7f016dd9b700 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc166070 0x5573cc168480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1d0166561b434a server_cookie=b6557fe2f52ae8d6 in_seq=0 out_seq=0 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.758+0000 7f0155ffb700 10 client.25652 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.759+0000 7f0155ffb700 1 -- 172.21.15.6:0/2496984625 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573cc069040 con 0x5573cc166070 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.759+0000 7f0155ffb700 10 client.25652 handle_command_reply: tid=0 2023-11-17T00:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.759+0000 7f01882c8b80 1 client.25652 shutdown 2023-11-17T00:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 msgr2=0x5573cc15ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0160009510 tx=0x7f0160004b80 comp rx=0 tx=0).stop 2023-11-17T00:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 shutdown_connections 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc166070 0x5573cc168480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cbfb16b0 0x5573cc161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 --2- 172.21.15.6:0/2496984625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cc155cf0 0x5573cc15ee40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 >> 172.21.15.6:0/2496984625 conn(0x5573cc14b1a0 msgr2=0x5573cc14fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f0155ffb700 0 client.25652 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 shutdown_connections 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.760+0000 7f01882c8b80 1 -- 172.21.15.6:0/2496984625 wait complete. 2023-11-17T00:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.761+0000 7f01882c8b80 20 client.25652 trim_cache size 0 max 16384 2023-11-17T00:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.787+0000 7f01882c8b80 1 Processor -- start 2023-11-17T00:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.787+0000 7f01882c8b80 1 -- start start 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc16cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc15d450 con 0x5573cc155cf0 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc159af0 con 0x5573cbfb16b0 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f01882c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc16d320 con 0x5573cc15b990 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f016e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc16cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:45736/0 (socket says 172.21.15.6:45736) 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f016dd9b700 1 -- 172.21.15.6:0/1096708539 learned_addr learned my addr 172.21.15.6:0/1096708539 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.788+0000 7f016dd9b700 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 -- 172.21.15.6:0/1096708539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 msgr2=0x5573cc150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 -- 172.21.15.6:0/1096708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 msgr2=0x5573cc1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 -- 172.21.15.6:0/1096708539 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573cc06ce80 con 0x5573cc155cf0 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f016e59c700 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc16cde0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5573cc1603f0 tx=0x7f0160006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beece7aa6b1b365 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f015d7fa700 1 -- 172.21.15.6:0/1096708539 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160015a30 con 0x5573cc155cf0 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f015d7fa700 1 -- 172.21.15.6:0/1096708539 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016000e030 con 0x5573cc155cf0 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.789+0000 7f015d7fa700 1 -- 172.21.15.6:0/1096708539 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160005040 con 0x5573cc155cf0 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 -- 172.21.15.6:0/1096708539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 msgr2=0x5573cc16cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc16cde0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5573cc1603f0 tx=0x7f0160006160 comp rx=0 tx=0).stop 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 -- 172.21.15.6:0/1096708539 shutdown_connections 2023-11-17T00:09:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc15b990 0x5573cc150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 --2- 172.21.15.6:0/1096708539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc155cf0 0x5573cc16cde0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.790+0000 7f01882c8b80 1 -- 172.21.15.6:0/1096708539 >> 172.21.15.6:0/1096708539 conn(0x5573cc14b1a0 msgr2=0x5573cc158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.791+0000 7f01882c8b80 1 -- 172.21.15.6:0/1096708539 shutdown_connections 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.791+0000 7f01882c8b80 1 -- 172.21.15.6:0/1096708539 wait complete. 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.791+0000 7f01882c8b80 1 -- start start 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.791+0000 7f01882c8b80 1 Processor -- start 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc155cf0 0x5573cc156e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc157530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc157c40 con 0x5573cc15b990 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc157db0 con 0x5573cbfb16b0 2023-11-17T00:09:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cc157f70 con 0x5573cc155cf0 2023-11-17T00:09:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f01882c8b80 10 client.0 _ll_register_callbacks cb 0x7f0164009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f016dd9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:45750/0 (socket says 172.21.15.6:45750) 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f016dd9b700 1 -- 172.21.15.6:0/4232828859 learned_addr learned my addr 172.21.15.6:0/4232828859 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.792+0000 7f016e59c700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc155cf0 0x5573cc156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 -- 172.21.15.6:0/4232828859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc155cf0 msgr2=0x5573cc156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc155cf0 0x5573cc156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 -- 172.21.15.6:0/4232828859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 msgr2=0x5573cc157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 -- 172.21.15.6:0/4232828859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573cc06d130 con 0x5573cbfb16b0 2023-11-17T00:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f016dd9b700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01701010d0 tx=0x7f0164020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852f6fa4c5d9c735 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.793+0000 7f0155ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016401e600 con 0x5573cbfb16b0 2023-11-17T00:09:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f016000f040 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0164004830 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0164003680 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 10 client.25658 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5573cbebdc00 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016000f040 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01640139d0 con 0x5573cbfb16b0 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 20 client.25658 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc1d4630 0x5573cc1d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 4 client.25658 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:09:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.794+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573cc075600 con 0x5573cc1d4630 2023-11-17T00:09:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.795+0000 7f016e59c700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc1d4630 0x5573cc1d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.795+0000 7f0155ffb700 10 client.25658 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.795+0000 7f016e59c700 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc1d4630 0x5573cc1d6a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14883e5aa74aac5f server_cookie=2e345262a148eaaa in_seq=0 out_seq=0 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f0155ffb700 1 -- 172.21.15.6:0/4232828859 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573cc075600 con 0x5573cc1d4630 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f0155ffb700 10 client.25658 handle_command_reply: tid=0 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 client.25658 shutdown 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 msgr2=0x5573cc16cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01701010d0 tx=0x7f0164020040 comp rx=0 tx=0).stop 2023-11-17T00:09:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 shutdown_connections 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5573cc1d4630 0x5573cc1d6a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5573cc15b990 0x5573cc157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5573cbfb16b0 0x5573cc16cd90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 --2- 172.21.15.6:0/4232828859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573cc155cf0 0x5573cc156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 >> 172.21.15.6:0/4232828859 conn(0x5573cc14b1a0 msgr2=0x5573cc15c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f0155ffb700 0 client.25658 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 shutdown_connections 2023-11-17T00:09:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 1 -- 172.21.15.6:0/4232828859 wait complete. 2023-11-17T00:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.803+0000 7f01882c8b80 20 client.25658 trim_cache size 0 max 16384 2023-11-17T00:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:09:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 msgr2=0x7f01580a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 0x7f01580a3910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01680cb390 tx=0x7f01680ce370 comp rx=0 tx=0).stop 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 msgr2=0x7f017019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 0x7f017019ccb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f016800f690 comp rx=0 tx=0).stop 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 shutdown_connections 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01580a1460 0x7f01580a3910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01701053a0 0x7f017019ccb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0170100360 0x7f0170198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 --2- 172.21.15.6:0/3440814480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0170100d00 0x7f01701988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 >> 172.21.15.6:0/3440814480 conn(0x7f01700fbbd0 msgr2=0x7f01700fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 shutdown_connections 2023-11-17T00:09:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:22.804+0000 7f016f59e700 1 -- 172.21.15.6:0/3440814480 wait complete. 2023-11-17T00:09:22.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:22 smithi087 ceph-mon[131182]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:22.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:22 smithi186 ceph-mon[131487]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:24.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:24 smithi006 ceph-mon[97840]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:24.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:24 smithi087 ceph-mon[131182]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:24.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:24 smithi186 ceph-mon[131487]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-11-17T00:09:26.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:26 smithi006 ceph-mon[97840]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-11-17T00:09:26.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:26 smithi087 ceph-mon[131182]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-11-17T00:09:26.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:26 smithi186 ceph-mon[131487]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-11-17T00:09:28.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:28 smithi006 ceph-mon[97840]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:09:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:28 smithi087 ceph-mon[131182]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:09:28.876 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:28 smithi186 ceph-mon[131487]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:09:30.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:30 smithi006 ceph-mon[97840]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2023-11-17T00:09:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:30 smithi087 ceph-mon[131182]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2023-11-17T00:09:30.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:30 smithi186 ceph-mon[131487]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2023-11-17T00:09:32.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:32 smithi006 ceph-mon[97840]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2023-11-17T00:09:32.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:32 smithi087 ceph-mon[131182]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2023-11-17T00:09:32.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:32 smithi186 ceph-mon[131487]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2023-11-17T00:09:34.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:34 smithi006 ceph-mon[97840]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:09:34.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:34 smithi087 ceph-mon[131182]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:09:34.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:34 smithi186 ceph-mon[131487]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 -- 172.21.15.6:0/1127477526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 msgr2=0x7ff4f0073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/1127477526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f0073200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009990 tx=0x7ff4ec009ca0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 -- 172.21.15.6:0/1127477526 shutdown_connections 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/1127477526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f0073200 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/1127477526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 0x7ff4f010eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/1127477526 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f010c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 -- 172.21.15.6:0/1127477526 >> 172.21.15.6:0/1127477526 conn(0x7ff4f00fbd10 msgr2=0x7ff4f00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 -- 172.21.15.6:0/1127477526 shutdown_connections 2023-11-17T00:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.958+0000 7ff4f81ff700 1 -- 172.21.15.6:0/1127477526 wait complete. 2023-11-17T00:09:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.959+0000 7ff4f81ff700 1 Processor -- start 2023-11-17T00:09:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 -- start start 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 0x7ff4f019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f019d5a0 con 0x7ff4f0072e30 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f019d710 con 0x7ff4f0073740 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f81ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f019d8d0 con 0x7ff4f010caf0 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f5f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f5f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49156/0 (socket says 172.21.15.6:49156) 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/2220654838 learned_addr learned my addr 172.21.15.6:0/2220654838 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f579a700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.960+0000 7ff4f679c700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 0x7ff4f019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/2220654838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 msgr2=0x7ff4f019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 0x7ff4f019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/2220654838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 msgr2=0x7ff4f0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/2220654838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0095f0 con 0x7ff4f0072e30 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f579a700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0095c0 tx=0x7ff4ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4e6ffd700 1 -- 172.21.15.6:0/2220654838 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec01d070 con 0x7ff4f0072e30 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f0112510 con 0x7ff4f0072e30 2023-11-17T00:09:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4e6ffd700 1 -- 172.21.15.6:0/2220654838 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec005050 con 0x7ff4f0072e30 2023-11-17T00:09:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.961+0000 7ff4e6ffd700 1 -- 172.21.15.6:0/2220654838 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec004a90 con 0x7ff4f0072e30 2023-11-17T00:09:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.962+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f0112380 con 0x7ff4f0072e30 2023-11-17T00:09:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.965+0000 7ff4e6ffd700 1 -- 172.21.15.6:0/2220654838 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff4ec005430 con 0x7ff4f0072e30 2023-11-17T00:09:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.966+0000 7ff50bb6ab80 1 Processor -- start 2023-11-17T00:09:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.966+0000 7ff50bb6ab80 1 -- start start 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4e6ffd700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 0x7ff4dc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b75dc000 con 0x5602b76dc720 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b762e840 con 0x5602b76dcb90 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b7627cd0 con 0x5602b75326b0 2023-11-17T00:09:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4e6ffd700 1 -- 172.21.15.6:0/2220654838 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff4ec0ce1d0 con 0x7ff4f0072e30 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49158/0 (socket says 172.21.15.6:49158) 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/4235018568 learned_addr learned my addr 172.21.15.6:0/4235018568 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f679c700 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f579a700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 0x7ff4dc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f679c700 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/4235018568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 msgr2=0x5602b76dc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76dc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/4235018568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 msgr2=0x5602b76d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.967+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/4235018568 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602b74a95d0 con 0x5602b76dc720 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.968+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec09acc0 tx=0x7ff4ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6f36b281268949 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.968+0000 7ff4f579a700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 0x7ff4dc0a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0199bd0 tx=0x7ff4e00073d0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.968+0000 7ff4dbfff700 1 -- 172.21.15.6:0/4235018568 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec09ad30 con 0x5602b76dc720 2023-11-17T00:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.968+0000 7ff4dbfff700 1 -- 172.21.15.6:0/4235018568 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec09ed60 con 0x5602b76dc720 2023-11-17T00:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.968+0000 7ff4dbfff700 1 -- 172.21.15.6:0/4235018568 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0a2ea0 con 0x5602b76dc720 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/4235018568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 msgr2=0x5602b76d6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec09acc0 tx=0x7ff4ec009310 comp rx=0 tx=0).stop 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/4235018568 shutdown_connections 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76dc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/4235018568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d6560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/4235018568 >> 172.21.15.6:0/4235018568 conn(0x5602b76cbf50 msgr2=0x5602b76d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/4235018568 shutdown_connections 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.969+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/4235018568 wait complete. 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.970+0000 7ff50bb6ab80 1 -- start start 2023-11-17T00:09:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.970+0000 7ff50bb6ab80 1 Processor -- start 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.970+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 0x5602b76e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.970+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b76dcb90 0x5602b76df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b75c9880 con 0x5602b76dc720 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b75c9520 con 0x5602b75326b0 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b74ab8c0 con 0x5602b76dcb90 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff50bb6ab80 10 client.0 _ll_register_callbacks cb 0x5602b75e7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49168/0 (socket says 172.21.15.6:49168) 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f579a700 1 -- 172.21.15.6:0/1187916981 learned_addr learned my addr 172.21.15.6:0/1187916981 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 0x5602b76e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f679c700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b76dcb90 0x5602b76df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/1187916981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b76dcb90 msgr2=0x5602b76df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b76dcb90 0x5602b76df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/1187916981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 msgr2=0x5602b76e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.971+0000 7ff4f5f9b700 1 -- 172.21.15.6:0/1187916981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f0112380 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 0x5602b76e5710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5602b76d71a0 tx=0x7ff4ec09a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6132c53f939e2ecd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0a2e60 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602b76c9710 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff4ec09d030 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec09ad30 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff50bb6ab80 10 client.25670 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5602b75ee130 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602b76c9710 con 0x5602b75326b0 2023-11-17T00:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.972+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4ec0b3330 con 0x5602b75326b0 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 10 client.25670 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 20 client.25670 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b76ef6d0 0x5602b76f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 4 client.25670 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602b75ea040 con 0x5602b76ef6d0 2023-11-17T00:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.973+0000 7ff4f579a700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b76ef6d0 0x5602b76f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.974+0000 7ff4d97fa700 10 client.25670 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.974+0000 7ff4f579a700 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b76ef6d0 0x5602b76f1ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e997b3dc26e68e7 server_cookie=e9203509de773428 in_seq=0 out_seq=0 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.974+0000 7ff4d97fa700 1 -- 172.21.15.6:0/1187916981 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5602b75ea040 con 0x5602b76ef6d0 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.974+0000 7ff4d97fa700 10 client.25670 handle_command_reply: tid=0 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.974+0000 7ff50bb6ab80 1 client.25670 shutdown 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 msgr2=0x5602b76e5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 0x5602b76e5710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5602b76d71a0 tx=0x7ff4ec09a5c0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 shutdown_connections 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b76dcb90 0x5602b76df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b75326b0 0x5602b76e5710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b76ef6d0 0x5602b76f1ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1187916981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 >> 172.21.15.6:0/1187916981 conn(0x5602b76cbf50 msgr2=0x5602b76d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff4d97fa700 0 client.25670 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 shutdown_connections 2023-11-17T00:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1187916981 wait complete. 2023-11-17T00:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.975+0000 7ff50bb6ab80 20 client.25670 trim_cache size 0 max 16384 2023-11-17T00:09:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.990+0000 7ff50bb6ab80 1 Processor -- start 2023-11-17T00:09:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.990+0000 7ff50bb6ab80 1 -- start start 2023-11-17T00:09:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b76d23e0 con 0x5602b76dc720 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b76ddf80 con 0x5602b76dcb90 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b76da6f0 con 0x5602b75326b0 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49182/0 (socket says 172.21.15.6:49182) 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 -- 172.21.15.6:0/1842106363 learned_addr learned my addr 172.21.15.6:0/1842106363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f679c700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 -- 172.21.15.6:0/1842106363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 msgr2=0x5602b76d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76d9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 -- 172.21.15.6:0/1842106363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 msgr2=0x5602b76d6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f579a700 1 -- 172.21.15.6:0/1842106363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602b75ede80 con 0x5602b76dc720 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f679c700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.991+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.992+0000 7ff4f579a700 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d6d0 tx=0x7ff4e001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfcb4083b2e2d6f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.992+0000 7ff4e4ff9700 1 -- 172.21.15.6:0/1842106363 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e002a040 con 0x5602b76dc720 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.992+0000 7ff4e4ff9700 1 -- 172.21.15.6:0/1842106363 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e0033070 con 0x5602b76dc720 2023-11-17T00:09:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.992+0000 7ff4e4ff9700 1 -- 172.21.15.6:0/1842106363 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e001b070 con 0x5602b76dc720 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1842106363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 msgr2=0x5602b76d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d6d0 tx=0x7ff4e001b000 comp rx=0 tx=0).stop 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1842106363 shutdown_connections 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b76d6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/1842106363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76d0f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1842106363 >> 172.21.15.6:0/1842106363 conn(0x5602b76cbf50 msgr2=0x5602b76e02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1842106363 shutdown_connections 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/1842106363 wait complete. 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 -- start start 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.993+0000 7ff50bb6ab80 1 Processor -- start 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b773c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b773c9d0 con 0x5602b76dc720 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b773cb40 con 0x5602b76dcb90 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602b773cd00 con 0x5602b75326b0 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff50bb6ab80 10 client.0 _ll_register_callbacks cb 0x7ff4e8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff4f579a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49184/0 (socket says 172.21.15.6:49184) 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff4f579a700 1 -- 172.21.15.6:0/3225781023 learned_addr learned my addr 172.21.15.6:0/3225781023 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.994+0000 7ff4f679c700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b773c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 -- 172.21.15.6:0/3225781023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 msgr2=0x5602b76e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76e3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 -- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 msgr2=0x5602b773c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b773c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 -- 172.21.15.6:0/3225781023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e0004400 con 0x5602b76dc720 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f679c700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b773c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4f579a700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0004120 tx=0x7ff4e002f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26960d28f01c73bb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e002a040 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff4e000dc00 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4ec09ed60 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4e000ddc0 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff50bb6ab80 10 client.15612 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.995+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5602b743ec00 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec09ed60 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4e0038720 con 0x5602b76dc720 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 10 client.15612 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 20 client.15612 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b7744480 0x5602b7746890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 4 client.15612 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5602b75f6600 con 0x5602b7744480 2023-11-17T00:09:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.996+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b7744480 0x5602b7746890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.997+0000 7ff4f5f9b700 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b7744480 0x5602b7746890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98789f354c58cde8 server_cookie=c1e07ff5ec62f4d2 in_seq=0 out_seq=0 2023-11-17T00:09:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:35.997+0000 7ff4d97fa700 10 client.15612 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff4d97fa700 1 -- 172.21.15.6:0/3225781023 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5602b75f6600 con 0x5602b7744480 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff4d97fa700 10 client.15612 handle_command_reply: tid=0 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 client.15612 shutdown 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 msgr2=0x5602b76e4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0004120 tx=0x7ff4e002f4f0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 shutdown_connections 2023-11-17T00:09:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5602b76dcb90 0x5602b773c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602b75326b0 0x5602b76e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5602b7744480 0x5602b7746890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 --2- 172.21.15.6:0/3225781023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5602b76dc720 0x5602b76e4030 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff4d97fa700 0 client.15612 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 >> 172.21.15.6:0/3225781023 conn(0x5602b76cbf50 msgr2=0x5602b76db390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 shutdown_connections 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 1 -- 172.21.15.6:0/3225781023 wait complete. 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.005+0000 7ff50bb6ab80 20 client.15612 trim_cache size 0 max 16384 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:09:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:09:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:09:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 msgr2=0x7ff4dc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 0x7ff4dc0a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0199bd0 tx=0x7ff4e00073d0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 msgr2=0x7ff4f01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0095c0 tx=0x7ff4ec00f690 comp rx=0 tx=0).stop 2023-11-17T00:09:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 shutdown_connections 2023-11-17T00:09:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4dc0a13e0 0x7ff4dc0a3890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4f0072e30 0x7ff4f01985c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4f010caf0 0x7ff4f019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 --2- 172.21.15.6:0/2220654838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f0073740 0x7ff4f0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 >> 172.21.15.6:0/2220654838 conn(0x7ff4f00fbd10 msgr2=0x7ff4f010eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 shutdown_connections 2023-11-17T00:09:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:36.006+0000 7ff4f81ff700 1 -- 172.21.15.6:0/2220654838 wait complete. 2023-11-17T00:09:36.725 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:36 smithi006 ceph-mon[97840]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:09:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:36 smithi087 ceph-mon[131182]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:09:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:36 smithi186 ceph-mon[131487]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-11-17T00:09:38.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:38 smithi006 ceph-mon[97840]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-11-17T00:09:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:38 smithi087 ceph-mon[131182]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-11-17T00:09:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:38 smithi186 ceph-mon[131487]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2023-11-17T00:09:39.336 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 67% (8149/12072) Updating files: 68% (8209/12072) Updating files: 69% (8330/12072) Updating files: 70% (8451/12072) Updating files: 71% (8572/12072) Updating files: 72% (8692/12072) Updating files: 73% (8813/12072) Updating files: 74% (8934/12072) Updating files: 75% (9054/12072) Updating files: 76% (9175/12072) Updating files: 77% (9296/12072) Updating files: 78% (9417/12072) Updating files: 79% (9537/12072) Updating files: 80% (9658/12072) Updating files: 81% (9779/12072) Updating files: 82% (9900/12072) Updating files: 83% (10020/12072) Updating files: 84% (10141/12072) Updating files: 85% (10262/12072) Updating files: 86% (10382/12072) Updating files: 87% (10503/12072) Updating files: 88% (10624/12072) Updating files: 89% (10745/12072) Updating files: 90% (10865/12072) Updating files: 91% (10986/12072) Updating files: 92% (11107/12072) Updating files: 93% (11227/12072) Updating files: 94% (11348/12072) Updating files: 95% (11469/12072) Updating files: 96% (11590/12072) Updating files: 97% (11710/12072) Updating files: 98% (11831/12072) Updating files: 99% (11952/12072) Updating files: 100% (12072/12072) Updating files: 100% (12072/12072), done. 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to '1c9565146c8647d4e00eb6c7ae626dc192680e7b'. 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2023-11-17T00:09:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at 1c9565146c8 Other1 2023-11-17T00:09:40.083 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-17T00:09:40.152 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-17T00:09:40.158 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-17T00:09:40.158 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-17T00:09:40.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:40 smithi006 ceph-mon[97840]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:09:40.757 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-17T00:09:40.843 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-17T00:09:40.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:40 smithi087 ceph-mon[131182]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:09:40.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:40 smithi186 ceph-mon[131487]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:09:40.908 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-17T00:09:40.912 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-17T00:09:40.912 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-17T00:09:40.975 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-17T00:09:40.983 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:09:40.983 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-17T00:09:41.030 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-17T00:09:41.032 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-17T00:09:41.032 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-17T00:09:41.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2023-11-17T00:09:41.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir -p testdir 2023-11-17T00:09:41.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd testdir 2023-11-17T00:09:41.136 INFO:tasks.workunit.client.0.smithi006.stderr:+ set +e 2023-11-17T00:09:41.137 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-17T00:09:41.194 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:09:41.194 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2023-11-17T00:09:41.194 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=0 2023-11-17T00:09:41.194 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 0 -lt 100 ']' 2023-11-17T00:09:41.194 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 0 + 1 2023-11-17T00:09:41.195 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=1 2023-11-17T00:09:41.195 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:41.205 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:41.206 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:41.207 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:41.207 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:41.208 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:42.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:42 smithi006 ceph-mon[97840]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:42 smithi087 ceph-mon[131182]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:42.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:42 smithi186 ceph-mon[131487]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:44.008 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:44.008 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:44.175 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:44.176 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:44.195 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:44.196 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:44.196 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -lt 100 ']' 2023-11-17T00:09:44.196 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 1 + 1 2023-11-17T00:09:44.227 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=2 2023-11-17T00:09:44.229 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:44.284 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:44.284 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:44.286 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:44.287 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:44.288 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:44.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:44 smithi006 ceph-mon[97840]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:44.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:44.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:44.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:44 smithi087 ceph-mon[131182]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:44.852 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:44.852 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:44.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:44 smithi186 ceph-mon[131487]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-11-17T00:09:44.868 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:44.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:44.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -lt 100 ']' 2023-11-17T00:09:44.869 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 2 + 1 2023-11-17T00:09:44.901 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=3 2023-11-17T00:09:44.901 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:44.933 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:44.933 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:44.936 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:44.937 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:44.937 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:45.197 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:45.197 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:45.223 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:45.223 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:45.231 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:45.231 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:45.231 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 3 -lt 100 ']' 2023-11-17T00:09:45.231 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 3 + 1 2023-11-17T00:09:45.235 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=4 2023-11-17T00:09:45.236 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:45.247 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:45.249 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:45.252 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:45.253 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:45.253 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:45.549 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:45.549 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:45.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:45.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:45.694 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:45.694 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:45.694 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 4 -lt 100 ']' 2023-11-17T00:09:45.694 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 4 + 1 2023-11-17T00:09:45.726 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=5 2023-11-17T00:09:45.726 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:45.762 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:45.763 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:45.764 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:45.764 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:45.765 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:46.166 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:46.166 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:46.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:46.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:46.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:46.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:46.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 5 -lt 100 ']' 2023-11-17T00:09:46.284 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 5 + 1 2023-11-17T00:09:46.311 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=6 2023-11-17T00:09:46.311 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:46.335 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:46.336 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:46.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:46.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:46.339 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:46.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:46.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:46.848 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:46.849 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:46.857 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:46 smithi087 ceph-mon[131182]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-17T00:09:46.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:46.873 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:46.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 6 -lt 100 ']' 2023-11-17T00:09:46.874 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 6 + 1 2023-11-17T00:09:46.875 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:46 smithi186 ceph-mon[131487]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-17T00:09:46.888 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=7 2023-11-17T00:09:46.888 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:46.910 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:46.910 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:46.913 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:46.913 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:46.914 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:46.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:46 smithi006 ceph-mon[97840]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-17T00:09:47.299 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:47.299 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:47.306 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:09:47.411 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:47.411 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:47.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:47.430 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:47.430 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 7 -lt 100 ']' 2023-11-17T00:09:47.430 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 7 + 1 2023-11-17T00:09:47.472 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=8 2023-11-17T00:09:47.472 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:47.497 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:47.497 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:47.501 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:47.502 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:47.518 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:47.999 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:47.999 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:48.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:48.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:48.087 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:48.087 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:48.087 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 8 -lt 100 ']' 2023-11-17T00:09:48.087 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 8 + 1 2023-11-17T00:09:48.124 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=9 2023-11-17T00:09:48.124 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:48.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:48.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:48.180 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:48.180 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:48.180 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:48.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:48.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:48.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:48 smithi087 ceph-mon[131182]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.6 KiB/s wr, 3 op/s 2023-11-17T00:09:48.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:48 smithi186 ceph-mon[131487]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.6 KiB/s wr, 3 op/s 2023-11-17T00:09:48.936 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:48.936 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:48.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:48.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:48.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 9 -lt 100 ']' 2023-11-17T00:09:48.946 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 9 + 1 2023-11-17T00:09:48.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:48 smithi006 ceph-mon[97840]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.6 KiB/s wr, 3 op/s 2023-11-17T00:09:49.005 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=10 2023-11-17T00:09:49.005 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:49.060 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:49.060 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:49.063 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:49.063 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:49.064 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:49.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:49.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:50.017 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:50.017 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:50.061 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:50.061 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:50.061 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 10 -lt 100 ']' 2023-11-17T00:09:50.061 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 10 + 1 2023-11-17T00:09:50.106 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=11 2023-11-17T00:09:50.106 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:50.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 -- 172.21.15.6:0/1978821928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5741015f0 msgr2=0x7ff574104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 --2- 172.21.15.6:0/1978821928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5741015f0 0x7ff574104e80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5640099c0 tx=0x7ff564009cd0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 -- 172.21.15.6:0/1978821928 shutdown_connections 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 --2- 172.21.15.6:0/1978821928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff574105560 0x7ff57410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 --2- 172.21.15.6:0/1978821928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574100c50 0x7ff574101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 --2- 172.21.15.6:0/1978821928 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5741015f0 0x7ff574104e80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 -- 172.21.15.6:0/1978821928 >> 172.21.15.6:0/1978821928 conn(0x7ff574077500 msgr2=0x7ff574077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 -- 172.21.15.6:0/1978821928 shutdown_connections 2023-11-17T00:09:50.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.111+0000 7ff57bbff700 1 -- 172.21.15.6:0/1978821928 wait complete. 2023-11-17T00:09:50.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.112+0000 7ff57bbff700 1 Processor -- start 2023-11-17T00:09:50.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 -- start start 2023-11-17T00:09:50.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 0x7ff574073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 0x7ff5741a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574073b60 con 0x7ff574100c50 2023-11-17T00:09:50.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741a4ac0 con 0x7ff574105560 2023-11-17T00:09:50.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57bbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741a4c00 con 0x7ff5741015f0 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57999b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57999b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54036/0 (socket says 172.21.15.6:54036) 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57999b700 1 -- 172.21.15.6:0/4164840332 learned_addr learned my addr 172.21.15.6:0/4164840332 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57a19c700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 0x7ff5741a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.113+0000 7ff57919a700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 0x7ff574073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 -- 172.21.15.6:0/4164840332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 msgr2=0x7ff574073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 0x7ff574073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 -- 172.21.15.6:0/4164840332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 msgr2=0x7ff5741a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 0x7ff5741a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 -- 172.21.15.6:0/4164840332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640095f0 con 0x7ff574100c50 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57919a700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 0x7ff574073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57a19c700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 0x7ff5741a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:50.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57999b700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9a0 tx=0x7ff57000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff56affd700 1 -- 172.21.15.6:0/4164840332 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570009790 con 0x7ff574100c50 2023-11-17T00:09:50.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff56affd700 1 -- 172.21.15.6:0/4164840332 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff57000cbd0 con 0x7ff574100c50 2023-11-17T00:09:50.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574110320 con 0x7ff574100c50 2023-11-17T00:09:50.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff56affd700 1 -- 172.21.15.6:0/4164840332 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570018680 con 0x7ff574100c50 2023-11-17T00:09:50.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.114+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574110190 con 0x7ff574100c50 2023-11-17T00:09:50.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.117+0000 7ff56affd700 1 -- 172.21.15.6:0/4164840332 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff57000cd40 con 0x7ff574100c50 2023-11-17T00:09:50.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.119+0000 7ff56affd700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 0x7ff5600a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.119+0000 7ff56affd700 1 -- 172.21.15.6:0/4164840332 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff570014070 con 0x7ff574100c50 2023-11-17T00:09:50.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.119+0000 7ff57919a700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 0x7ff5600a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.119+0000 7ff57919a700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 0x7ff5600a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5640098e0 tx=0x7ff56401a040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 Processor -- start 2023-11-17T00:09:50.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 -- start start 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affa7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afea8060 con 0x55a3affa3470 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afefa8a0 con 0x55a3affa2b60 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.184+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afef3d30 con 0x55a3affa78b0 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57919a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57919a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54042/0 (socket says 172.21.15.6:54042) 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57919a700 1 -- 172.21.15.6:0/3943237787 learned_addr learned my addr 172.21.15.6:0/3943237787 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57a19c700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affa7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 -- 172.21.15.6:0/3943237787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 msgr2=0x55a3affa7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affa7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 -- 172.21.15.6:0/3943237787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 msgr2=0x55a3affa7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 -- 172.21.15.6:0/3943237787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3afd75630 con 0x55a3affa2b60 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57919a700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.185+0000 7ff57999b700 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa2f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5700ca920 tx=0x7ff57009a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de67d8196db347f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff55ffff700 1 -- 172.21.15.6:0/3943237787 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700a2620 con 0x55a3affa2b60 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff55ffff700 1 -- 172.21.15.6:0/3943237787 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5700cd300 con 0x55a3affa2b60 2023-11-17T00:09:50.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff55ffff700 1 -- 172.21.15.6:0/3943237787 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700a2460 con 0x55a3affa2b60 2023-11-17T00:09:50.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 -- 172.21.15.6:0/3943237787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 msgr2=0x55a3affa2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa2f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5700ca920 tx=0x7ff57009a1d0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 -- 172.21.15.6:0/3943237787 shutdown_connections 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affa7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa2f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/3943237787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa3470 0x55a3affa7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.186+0000 7ff58f60ab80 1 -- 172.21.15.6:0/3943237787 >> 172.21.15.6:0/3943237787 conn(0x55a3aff98360 msgr2=0x55a3aff9db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.187+0000 7ff58f60ab80 1 -- 172.21.15.6:0/3943237787 shutdown_connections 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.187+0000 7ff58f60ab80 1 -- 172.21.15.6:0/3943237787 wait complete. 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.187+0000 7ff58f60ab80 1 -- start start 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.187+0000 7ff58f60ab80 1 Processor -- start 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa3470 0x55a3affac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afe958e0 con 0x55a3affa2b60 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afe95580 con 0x55a3affa3470 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3afd77920 con 0x55a3affa78b0 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff58f60ab80 10 client.0 _ll_register_callbacks cb 0x55a3afeb3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54058/0 (socket says 172.21.15.6:54058) 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 -- 172.21.15.6:0/4245849770 learned_addr learned my addr 172.21.15.6:0/4245849770 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57919a700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa3470 0x55a3affac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57a19c700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 -- 172.21.15.6:0/4245849770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 msgr2=0x55a3affac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 -- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa3470 msgr2=0x55a3affac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa3470 0x55a3affac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.188+0000 7ff57999b700 1 -- 172.21.15.6:0/4245849770 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574110190 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff57999b700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff57009a830 tx=0x7ff570096510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ed0ef931327450 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570098570 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff5700998d0 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3aff95b20 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5700b2bb0 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff58f60ab80 10 client.15624 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a3afeba190 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff570064080 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.189+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff570099130 con 0x55a3affa2b60 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 10 client.15624 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:50.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 20 client.15624 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3affbc410 0x55a3affbe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 4 client.15624 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3afeb60a0 con 0x55a3affbc410 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff57919a700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3affbc410 0x55a3affbe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff57919a700 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3affbc410 0x55a3affbe7f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3482ec2430b5c7 server_cookie=abd8de13a656e999 in_seq=0 out_seq=0 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.190+0000 7ff55d7fa700 10 client.15624 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff55d7fa700 1 -- 172.21.15.6:0/4245849770 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a3afeb60a0 con 0x55a3affbc410 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff55d7fa700 10 client.15624 handle_command_reply: tid=0 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 client.15624 shutdown 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 msgr2=0x55a3affaf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff57009a830 tx=0x7ff570096510 comp rx=0 tx=0).stop 2023-11-17T00:09:50.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 shutdown_connections 2023-11-17T00:09:50.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3affbc410 0x55a3affbe7f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa2b60 0x55a3affaf040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa78b0 0x55a3affac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff55d7fa700 0 client.15624 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/4245849770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa3470 0x55a3affac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.191+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 >> 172.21.15.6:0/4245849770 conn(0x55a3aff98360 msgr2=0x55a3aff9cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.192+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 shutdown_connections 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.192+0000 7ff58f60ab80 1 -- 172.21.15.6:0/4245849770 wait complete. 2023-11-17T00:09:50.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.192+0000 7ff58f60ab80 20 client.15624 trim_cache size 0 max 16384 2023-11-17T00:09:50.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.209+0000 7ff58f60ab80 1 Processor -- start 2023-11-17T00:09:50.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.209+0000 7ff58f60ab80 1 -- start start 2023-11-17T00:09:50.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affa51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affa3980 con 0x55a3affa78b0 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affa5c20 con 0x55a3affa2b60 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affad870 con 0x55a3affa3470 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54068/0 (socket says 172.21.15.6:54068) 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 -- 172.21.15.6:0/793449670 learned_addr learned my addr 172.21.15.6:0/793449670 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57999b700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57919a700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affa51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 -- 172.21.15.6:0/793449670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 msgr2=0x55a3affa51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affa51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 -- 172.21.15.6:0/793449670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 msgr2=0x55a3affa5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57a19c700 1 -- 172.21.15.6:0/793449670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3aff99b00 con 0x55a3affa78b0 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57999b700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.210+0000 7ff57919a700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affa51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.211+0000 7ff57a19c700 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f20 tx=0x7ff56c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b92a52e82ebef78 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.211+0000 7ff568bff700 1 -- 172.21.15.6:0/793449670 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003c00 con 0x55a3affa78b0 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.211+0000 7ff568bff700 1 -- 172.21.15.6:0/793449670 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff56c00f040 con 0x55a3affa78b0 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.211+0000 7ff568bff700 1 -- 172.21.15.6:0/793449670 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c017770 con 0x55a3affa78b0 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- 172.21.15.6:0/793449670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 msgr2=0x55a3affa56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f20 tx=0x7ff56c007700 comp rx=0 tx=0).stop 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- 172.21.15.6:0/793449670 shutdown_connections 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affa56e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affa5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/793449670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affa51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- 172.21.15.6:0/793449670 >> 172.21.15.6:0/793449670 conn(0x55a3aff98360 msgr2=0x55a3affa7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- 172.21.15.6:0/793449670 shutdown_connections 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- 172.21.15.6:0/793449670 wait complete. 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.212+0000 7ff58f60ab80 1 -- start start 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 Processor -- start 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affb19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affb3f90 con 0x55a3affa78b0 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affb2450 con 0x55a3affa2b60 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3affb25e0 con 0x55a3affa3470 2023-11-17T00:09:50.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.213+0000 7ff58f60ab80 10 client.0 _ll_register_callbacks cb 0x7ff564005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42002/0 (socket says 172.21.15.6:42002) 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 -- 172.21.15.6:0/2221962989 learned_addr learned my addr 172.21.15.6:0/2221962989 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57919a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affb19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57a19c700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 -- 172.21.15.6:0/2221962989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 msgr2=0x55a3affb19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affb19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 -- 172.21.15.6:0/2221962989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 msgr2=0x55a3affb1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affb1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 -- 172.21.15.6:0/2221962989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c009bd0 con 0x55a3affa2b60 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57919a700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affb19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff57999b700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5700994c0 tx=0x7ff5700ca860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d5e743d32dff2c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.214+0000 7ff55d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700b2bb0 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff5700b2d10 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5700cd300 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5700b82d0 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 10 client.25694 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a3afd21de0 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5700cd300 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff570095fc0 con 0x55a3affa2b60 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 10 client.25694 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 20 client.25694 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3b00110e0 0x55a3b00134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 4 client.25694 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:09:50.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.215+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3afec2660 con 0x55a3b00110e0 2023-11-17T00:09:50.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff57919a700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3b00110e0 0x55a3b00134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.267 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:50.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff57919a700 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3b00110e0 0x55a3b00134c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a15885f0f0264de server_cookie=b4b1ca94b611949c in_seq=0 out_seq=0 2023-11-17T00:09:50.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff55d7fa700 10 client.25694 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff55d7fa700 1 -- 172.21.15.6:0/2221962989 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a3afec2660 con 0x55a3b00110e0 2023-11-17T00:09:50.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff55d7fa700 10 client.25694 handle_command_reply: tid=0 2023-11-17T00:09:50.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.216+0000 7ff58f60ab80 1 client.25694 shutdown 2023-11-17T00:09:50.268 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 msgr2=0x55a3affb36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5700994c0 tx=0x7ff5700ca860 comp rx=0 tx=0).stop 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 shutdown_connections 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a3b00110e0 0x55a3b00134c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a3affa78b0 0x55a3affb1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a3affa2b60 0x55a3affb36c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 --2- 172.21.15.6:0/2221962989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a3affa3470 0x55a3affb19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff55d7fa700 0 client.25694 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 >> 172.21.15.6:0/2221962989 conn(0x55a3aff98360 msgr2=0x55a3affa5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 shutdown_connections 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 1 -- 172.21.15.6:0/2221962989 wait complete. 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.217+0000 7ff58f60ab80 20 client.25694 trim_cache size 0 max 16384 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:09:50.270 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 msgr2=0x7ff5600a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 0x7ff5600a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5640098e0 tx=0x7ff56401a040 comp rx=0 tx=0).stop 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 msgr2=0x7ff574072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9a0 tx=0x7ff57000ed60 comp rx=0 tx=0).stop 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 shutdown_connections 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff5600a13e0 0x7ff5600a3890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff574100c50 0x7ff574072df0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff574105560 0x7ff5741a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 --2- 172.21.15.6:0/4164840332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5741015f0 0x7ff574073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 >> 172.21.15.6:0/4164840332 conn(0x7ff574077500 msgr2=0x7ff5740feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 shutdown_connections 2023-11-17T00:09:50.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:50.218+0000 7ff57bbff700 1 -- 172.21.15.6:0/4164840332 wait complete. 2023-11-17T00:09:50.272 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:50.272 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:50.272 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:50.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:09:50.273 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:09:50.606 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:50 smithi006 ceph-mon[97840]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 -- 172.21.15.6:0/3511568239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 msgr2=0x7f2dbc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/3511568239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc104e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0099c0 tx=0x7f2dac009cd0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 -- 172.21.15.6:0/3511568239 shutdown_connections 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/3511568239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc10d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/3511568239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 0x7f2dbc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/3511568239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc104e00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 -- 172.21.15.6:0/3511568239 >> 172.21.15.6:0/3511568239 conn(0x7f2dbc0fbd10 msgr2=0x7f2dbc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 -- 172.21.15.6:0/3511568239 shutdown_connections 2023-11-17T00:09:50.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.634+0000 7f2dc3fff700 1 -- 172.21.15.6:0/3511568239 wait complete. 2023-11-17T00:09:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 Processor -- start 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 -- start start 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 0x7f2dbc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc073b60 con 0x7f2dbc100e40 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1a4af0 con 0x7f2dbc1054e0 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.635+0000 7f2dc3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1a4c30 con 0x7f2dbc1004a0 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 0x7f2dbc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54100/0 (socket says 172.21.15.6:54100) 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 -- 172.21.15.6:0/2374705704 learned_addr learned my addr 172.21.15.6:0/2374705704 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 -- 172.21.15.6:0/2374705704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 msgr2=0x7f2dbc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 0x7f2dbc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 -- 172.21.15.6:0/2374705704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 msgr2=0x7f2dbc1a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc1a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 -- 172.21.15.6:0/2374705704 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0095f0 con 0x7f2dbc100e40 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.636+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098e0 tx=0x7f2dac00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.637+0000 7f2db2ffd700 1 -- 172.21.15.6:0/2374705704 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac01d070 con 0x7f2dbc100e40 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.637+0000 7f2db2ffd700 1 -- 172.21.15.6:0/2374705704 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dac022470 con 0x7f2dbc100e40 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.637+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc1102b0 con 0x7f2dbc100e40 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.637+0000 7f2db2ffd700 1 -- 172.21.15.6:0/2374705704 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac017650 con 0x7f2dbc100e40 2023-11-17T00:09:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.637+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc110120 con 0x7f2dbc100e40 2023-11-17T00:09:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.640+0000 7f2db2ffd700 1 -- 172.21.15.6:0/2374705704 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2dac022850 con 0x7f2dbc100e40 2023-11-17T00:09:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.641+0000 7f2db2ffd700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 0x7f2da80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.642+0000 7f2db2ffd700 1 -- 172.21.15.6:0/2374705704 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2dac0ce1b0 con 0x7f2dbc100e40 2023-11-17T00:09:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.642+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 0x7f2da80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.642+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 0x7f2da80a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009c60 tx=0x7f2db8009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.832+0000 7f2dd78b2b80 1 Processor -- start 2023-11-17T00:09:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.832+0000 7f2dd78b2b80 1 -- start start 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 0x55e311c7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311b7c000 con 0x55e311c7c720 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311bce840 con 0x55e311c7cb90 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311bc7cd0 con 0x55e311ad26b0 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dc159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dc159a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54112/0 (socket says 172.21.15.6:54112) 2023-11-17T00:09:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dc159a700 1 -- 172.21.15.6:0/2869132259 learned_addr learned my addr 172.21.15.6:0/2869132259 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 0x55e311c7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.833+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 -- 172.21.15.6:0/2869132259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 msgr2=0x55e311c7c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 0x55e311c7c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 -- 172.21.15.6:0/2869132259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 msgr2=0x55e311c76b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 -- 172.21.15.6:0/2869132259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e311a495d0 con 0x55e311c7c720 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 0x55e311c7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0a2ba0 tx=0x7f2dac097150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beb1c7b6189cc7d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2da7fff700 1 -- 172.21.15.6:0/2869132259 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac09ad10 con 0x55e311c7c720 2023-11-17T00:09:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2da7fff700 1 -- 172.21.15.6:0/2869132259 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dac09ed40 con 0x55e311c7c720 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.834+0000 7f2da7fff700 1 -- 172.21.15.6:0/2869132259 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac09a460 con 0x55e311c7c720 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2869132259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 msgr2=0x55e311c76560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0a2ba0 tx=0x7f2dac097150 comp rx=0 tx=0).stop 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2869132259 shutdown_connections 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311ad26b0 0x55e311c7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2869132259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7c720 0x55e311c76560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2869132259 >> 172.21.15.6:0/2869132259 conn(0x55e311c6bf50 msgr2=0x55e311c71270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.835+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2869132259 shutdown_connections 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.836+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2869132259 wait complete. 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.836+0000 7f2dd78b2b80 1 -- start start 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.836+0000 7f2dd78b2b80 1 Processor -- start 2023-11-17T00:09:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311ad26b0 0x55e311c85760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c85ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311b69880 con 0x55e311c7cb90 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311b69520 con 0x55e311ad26b0 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311a4b8c0 con 0x55e311c7c720 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dd78b2b80 10 client.0 _ll_register_callbacks cb 0x55e311b87720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54122/0 (socket says 172.21.15.6:54122) 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 -- 172.21.15.6:0/1010708629 learned_addr learned my addr 172.21.15.6:0/1010708629 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311ad26b0 0x55e311c85760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc159a700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c85ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 -- 172.21.15.6:0/1010708629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 msgr2=0x55e311c85ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c85ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.837+0000 7f2dc259c700 1 -- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311ad26b0 msgr2=0x55e311c85760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dc259c700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311ad26b0 0x55e311c85760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dc259c700 1 -- 172.21.15.6:0/1010708629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc110120 con 0x55e311c7cb90 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dc159a700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c85ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dc259c700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2db40047a0 tx=0x7f2db400e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50d455920566439 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2da57fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40092d0 con 0x55e311c7cb90 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2db4007500 con 0x55e311c7cb90 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e311c69710 con 0x55e311c7cb90 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db40048e0 con 0x55e311c7cb90 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dd78b2b80 10 client.15642 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.838+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e311b8e130 con 0x55e311c7cb90 2023-11-17T00:09:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.839+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e311c69710 con 0x55e311c7cb90 2023-11-17T00:09:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.839+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2db401c020 con 0x55e311c7cb90 2023-11-17T00:09:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.839+0000 7f2dd78b2b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:50.916 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:50 smithi186 ceph-mon[131487]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2023-11-17T00:09:50.916 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:50 smithi087 ceph-mon[131182]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.860+0000 7f2dd78b2b80 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.860+0000 7f2dd78b2b80 20 client.15642 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.861+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311c8fc50 0x55e311c92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.861+0000 7f2dd78b2b80 4 client.15642 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.861+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e311b8a040 con 0x55e311c8fc50 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.861+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311c8fc50 0x55e311c92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.862+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311c8fc50 0x55e311c92030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545b6cff2b355bd6 server_cookie=dabb29cf008273b0 in_seq=0 out_seq=0 2023-11-17T00:09:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.862+0000 7f2da57fa700 10 client.15642 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.862+0000 7f2da57fa700 1 -- 172.21.15.6:0/1010708629 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e311b8a040 con 0x55e311c8fc50 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.862+0000 7f2da57fa700 10 client.15642 handle_command_reply: tid=0 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 client.15642 shutdown 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 msgr2=0x55e311c7fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2db40047a0 tx=0x7f2db400e7d0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 shutdown_connections 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311c8fc50 0x55e311c92030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311c7cb90 0x55e311c7fa60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311ad26b0 0x55e311c85760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2da57fa700 0 client.15642 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/1010708629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c85ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 >> 172.21.15.6:0/1010708629 conn(0x55e311c6bf50 msgr2=0x55e311c707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 shutdown_connections 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/1010708629 wait complete. 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.863+0000 7f2dd78b2b80 20 client.15642 trim_cache size 0 max 16384 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 Processor -- start 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 -- start start 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c70f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311c8d360 con 0x55e311ad26b0 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311c71d60 con 0x55e311c7cb90 2023-11-17T00:09:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.879+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311c721c0 con 0x55e311c7c720 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54136/0 (socket says 172.21.15.6:54136) 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/3136179554 learned_addr learned my addr 172.21.15.6:0/3136179554 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc159a700 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c70f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/3136179554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 msgr2=0x55e311c70f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c70f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/3136179554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 msgr2=0x55e311c76560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/3136179554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e311c6d6f0 con 0x55e311ad26b0 2023-11-17T00:09:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc259c700 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e311c92940 tx=0x7f2db8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab48f5e5d9092ed server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.880+0000 7f2db0ff9700 1 -- 172.21.15.6:0/3136179554 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8022030 con 0x55e311ad26b0 2023-11-17T00:09:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2db0ff9700 1 -- 172.21.15.6:0/3136179554 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db8023040 con 0x55e311ad26b0 2023-11-17T00:09:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2db0ff9700 1 -- 172.21.15.6:0/3136179554 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db801dd60 con 0x55e311ad26b0 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/3136179554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 msgr2=0x55e311c7fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e311c92940 tx=0x7f2db8014000 comp rx=0 tx=0).stop 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/3136179554 shutdown_connections 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c7fa10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7cb90 0x55e311c76560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/3136179554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7c720 0x55e311c70f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/3136179554 >> 172.21.15.6:0/3136179554 conn(0x55e311c6bf50 msgr2=0x55e311c7f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/3136179554 shutdown_connections 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.881+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/3136179554 wait complete. 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.882+0000 7f2dd78b2b80 1 -- start start 2023-11-17T00:09:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.882+0000 7f2dd78b2b80 1 Processor -- start 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7c720 0x55e311c83e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 0x55e311ce2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311ce3020 con 0x55e311ad26b0 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311ce3160 con 0x55e311c7c720 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e311ce32d0 con 0x55e311c7cb90 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dd78b2b80 10 client.0 _ll_register_callbacks cb 0x7f2db40040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54152/0 (socket says 172.21.15.6:54152) 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/2254830058 learned_addr learned my addr 172.21.15.6:0/2254830058 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7c720 0x55e311c83e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 0x55e311ce2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/2254830058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 msgr2=0x55e311ce2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 0x55e311ce2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7c720 msgr2=0x55e311c83e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.883+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7c720 0x55e311c83e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dc1d9b700 1 -- 172.21.15.6:0/2254830058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e311b8e130 con 0x55e311ad26b0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dc259c700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 0x55e311ce2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dc1d9b700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2db8011ab0 tx=0x7f2db802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724caf53c7e467c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2da57fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8022040 con 0x55e311ad26b0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2db802b3f0 con 0x55e311ad26b0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dac09ed40 con 0x55e311ad26b0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2db802b5b0 con 0x55e311ad26b0 2023-11-17T00:09:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dd78b2b80 10 client.15648 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2db802bab0 con 0x55e311ad26b0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.884+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dac09ed40 con 0x55e311ad26b0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2db8032070 con 0x55e311ad26b0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 20 client.15648 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311ce49b0 0x55e311ce6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 4 client.15648 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e311b96600 con 0x55e311ce49b0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311ce49b0 0x55e311ce6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2dc159a700 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311ce49b0 0x55e311ce6dc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f4c5d243ff0145 server_cookie=b4932302fd756e48 in_seq=0 out_seq=0 2023-11-17T00:09:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.885+0000 7f2da57fa700 10 client.15648 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.944 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:50.944 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2da57fa700 1 -- 172.21.15.6:0/2254830058 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e311b96600 con 0x55e311ce49b0 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2da57fa700 10 client.15648 handle_command_reply: tid=0 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 client.15648 shutdown 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 msgr2=0x55e311c83950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2db8011ab0 tx=0x7f2db802be20 comp rx=0 tx=0).stop 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 shutdown_connections 2023-11-17T00:09:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e311ce49b0 0x55e311ce6dc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e311ad26b0 0x55e311c83950 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e311c7cb90 0x55e311ce2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 --2- 172.21.15.6:0/2254830058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e311c7c720 0x55e311c83e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2da57fa700 0 client.15648 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 >> 172.21.15.6:0/2254830058 conn(0x55e311c6bf50 msgr2=0x55e311c7e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 shutdown_connections 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.900+0000 7f2dd78b2b80 1 -- 172.21.15.6:0/2254830058 wait complete. 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dd78b2b80 20 client.15648 trim_cache size 0 max 16384 2023-11-17T00:09:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:09:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:09:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:09:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 msgr2=0x7f2da80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 0x7f2da80a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009c60 tx=0x7f2db8009260 comp rx=0 tx=0).stop 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 msgr2=0x7f2dbc073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098e0 tx=0x7f2dac00f690 comp rx=0 tx=0).stop 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 shutdown_connections 2023-11-17T00:09:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2da80a13e0 0x7f2da80a3890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dbc1054e0 0x7f2dbc1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2dbc1004a0 0x7f2dbc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 --2- 172.21.15.6:0/2374705704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2dbc100e40 0x7f2dbc073330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 >> 172.21.15.6:0/2374705704 conn(0x7f2dbc0fbd10 msgr2=0x7f2dbc0fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 shutdown_connections 2023-11-17T00:09:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:09:50.901+0000 7f2dc3fff700 1 -- 172.21.15.6:0/2374705704 wait complete. 2023-11-17T00:09:51.075 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:51.075 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:51.085 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:51.085 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:51.085 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 11 -lt 100 ']' 2023-11-17T00:09:51.315 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 11 + 1 2023-11-17T00:09:51.317 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=12 2023-11-17T00:09:51.317 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:51.317 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:51.318 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:51.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:51.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:51.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:51.697 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:51.698 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:51.936 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:51.936 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:51.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:51.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:51.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 12 -lt 100 ']' 2023-11-17T00:09:51.976 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 12 + 1 2023-11-17T00:09:52.047 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=13 2023-11-17T00:09:52.047 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:52.151 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:52.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:52.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:52.155 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:52.155 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:52.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:52.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:52.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:52 smithi087 ceph-mon[131182]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:52.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:52 smithi186 ceph-mon[131487]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:52.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:52 smithi006 ceph-mon[97840]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 -- 172.21.15.6:0/3262852834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40100cc0 msgr2=0x7fbf40101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 --2- 172.21.15.6:0/3262852834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40100cc0 0x7fbf40101090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf300098f0 tx=0x7fbf30009c00 comp rx=0 tx=0).stop 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 -- 172.21.15.6:0/3262852834 shutdown_connections 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 --2- 172.21.15.6:0/3262852834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40105620 0x7fbf4010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 --2- 172.21.15.6:0/3262852834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40100cc0 0x7fbf40101090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 --2- 172.21.15.6:0/3262852834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 -- 172.21.15.6:0/3262852834 >> 172.21.15.6:0/3262852834 conn(0x7fbf400774d0 msgr2=0x7fbf400778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:53.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 -- 172.21.15.6:0/3262852834 shutdown_connections 2023-11-17T00:09:53.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.086+0000 7fbf453ff700 1 -- 172.21.15.6:0/3262852834 wait complete. 2023-11-17T00:09:53.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.088+0000 7fbf453ff700 1 Processor -- start 2023-11-17T00:09:53.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.088+0000 7fbf453ff700 1 -- start start 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.088+0000 7fbf453ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.088+0000 7fbf453ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.088+0000 7fbf453ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40105620 0x7fbf4019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf453ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019c610 con 0x7fbf40101660 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf453ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019c780 con 0x7fbf40100cc0 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf453ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019c940 con 0x7fbf40105620 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42060/0 (socket says 172.21.15.6:42060) 2023-11-17T00:09:53.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3effd700 1 -- 172.21.15.6:0/1332105618 learned_addr learned my addr 172.21.15.6:0/1332105618 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3f7fe700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40105620 0x7fbf4019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 -- 172.21.15.6:0/1332105618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40105620 msgr2=0x7fbf4019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40105620 0x7fbf4019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 -- 172.21.15.6:0/1332105618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 msgr2=0x7fbf40198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3e7fc700 1 -- 172.21.15.6:0/1332105618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf40101660 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.089+0000 7fbf3effd700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:53.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf3e7fc700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40198c60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf40101a30 tx=0x7fbf3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:53.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400cba0 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf30009520 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf340044e0 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003dc0 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.090+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf401103d0 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.091+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf40110240 con 0x7fbf40101660 2023-11-17T00:09:53.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.093+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fbf34003f60 con 0x7fbf40101660 2023-11-17T00:09:53.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.095+0000 7fbf27fff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 0x7fbf280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:53.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.095+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf34014070 con 0x7fbf40101660 2023-11-17T00:09:53.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.095+0000 7fbf3effd700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 0x7fbf280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:53.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.096+0000 7fbf3effd700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 0x7fbf280a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf30011040 tx=0x7fbf30019040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:53.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.117+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf34096580 con 0x7fbf40101660 2023-11-17T00:09:53.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:53.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:53.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:53.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:53.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 13 -lt 100 ']' 2023-11-17T00:09:53.153 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 13 + 1 2023-11-17T00:09:53.183 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=14 2023-11-17T00:09:53.183 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:53.218 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:53.219 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:53.221 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:53.221 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:53.222 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.382+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf40110560 con 0x7fbf40101660 2023-11-17T00:09:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.382+0000 7fbf27fff700 1 -- 172.21.15.6:0/1332105618 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7fbf34099ba0 con 0x7fbf40101660 2023-11-17T00:09:53.384 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:09:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:09:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 msgr2=0x7fbf280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 0x7fbf280a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf30011040 tx=0x7fbf30019040 comp rx=0 tx=0).stop 2023-11-17T00:09:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 msgr2=0x7fbf40198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40198c60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf40101a30 tx=0x7fbf3400ee70 comp rx=0 tx=0).stop 2023-11-17T00:09:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 shutdown_connections 2023-11-17T00:09:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf280a13e0 0x7fbf280a3890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf40105620 0x7fbf4019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf40100cc0 0x7fbf40198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 --2- 172.21.15.6:0/1332105618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf40101660 0x7fbf40198c60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 >> 172.21.15.6:0/1332105618 conn(0x7fbf400774d0 msgr2=0x7fbf401001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 shutdown_connections 2023-11-17T00:09:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:53.385+0000 7fbf453ff700 1 -- 172.21.15.6:0/1332105618 wait complete. 2023-11-17T00:09:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:09:53.447 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:09:53.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:53 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1332105618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:53.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:53 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1332105618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:53.856 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:53.857 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:53.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:53 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1332105618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:09:53.867 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:53.867 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:53.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:53.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:53.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 14 -lt 100 ']' 2023-11-17T00:09:53.884 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 14 + 1 2023-11-17T00:09:53.921 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=15 2023-11-17T00:09:53.922 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:53.937 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:53.937 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:53.940 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:53.940 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:53.941 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:54.121 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:54.121 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:54.277 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:54.277 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:54.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:54.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:54.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 15 -lt 100 ']' 2023-11-17T00:09:54.286 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 15 + 1 2023-11-17T00:09:54.291 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=16 2023-11-17T00:09:54.353 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:54.353 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:54.354 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:54.354 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:54.354 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:54.354 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:54.777 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:54 smithi006 ceph-mon[97840]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:54 smithi087 ceph-mon[131182]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:54.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:54 smithi186 ceph-mon[131487]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2023-11-17T00:09:55.006 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:55.006 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:55.249 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:55.249 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:55.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:55.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:55.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 16 -lt 100 ']' 2023-11-17T00:09:55.268 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 16 + 1 2023-11-17T00:09:55.313 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=17 2023-11-17T00:09:55.313 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:55.342 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:55.343 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:55.345 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:55.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:55.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 -- 172.21.15.6:0/2392829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8510100c50 msgr2=0x7f8510101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 --2- 172.21.15.6:0/2392829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8510100c50 0x7f8510101020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8500009920 tx=0x7f8500009c30 comp rx=0 tx=0).stop 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 -- 172.21.15.6:0/2392829163 shutdown_connections 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 --2- 172.21.15.6:0/2392829163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8510100c50 0x7f8510101020 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 --2- 172.21.15.6:0/2392829163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f851010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 --2- 172.21.15.6:0/2392829163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85101015f0 0x7f8510104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 -- 172.21.15.6:0/2392829163 >> 172.21.15.6:0/2392829163 conn(0x7f8510077590 msgr2=0x7f8510077990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 -- 172.21.15.6:0/2392829163 shutdown_connections 2023-11-17T00:09:55.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.779+0000 7f85161ff700 1 -- 172.21.15.6:0/2392829163 wait complete. 2023-11-17T00:09:55.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.794+0000 7f85161ff700 1 Processor -- start 2023-11-17T00:09:55.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 -- start start 2023-11-17T00:09:55.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 0x7f8510072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 0x7f8510073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f85101a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510073b60 con 0x7f85101015f0 2023-11-17T00:09:55.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85101a4b70 con 0x7f85101055c0 2023-11-17T00:09:55.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f85161ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85101a4cb0 con 0x7f8510100c50 2023-11-17T00:09:55.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 0x7f8510073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f850ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f85101a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 0x7f8510072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 0x7f8510073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54190/0 (socket says 172.21.15.6:54190) 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f8507fff700 1 -- 172.21.15.6:0/3380525252 learned_addr learned my addr 172.21.15.6:0/3380525252 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.795+0000 7f850ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f85101a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42088/0 (socket says 172.21.15.6:42088) 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 -- 172.21.15.6:0/3380525252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 msgr2=0x7f85101a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f85101a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 -- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 msgr2=0x7f8510073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 0x7f8510073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 -- 172.21.15.6:0/3380525252 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500009520 con 0x7f8510100c50 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850f7fe700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 0x7f8510072df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8500009230 tx=0x7f850000fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850d7fa700 1 -- 172.21.15.6:0/3380525252 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001c070 con 0x7f8510100c50 2023-11-17T00:09:55.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8510110380 con 0x7f8510100c50 2023-11-17T00:09:55.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850d7fa700 1 -- 172.21.15.6:0/3380525252 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850000bb20 con 0x7f8510100c50 2023-11-17T00:09:55.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.796+0000 7f850d7fa700 1 -- 172.21.15.6:0/3380525252 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500003900 con 0x7f8510100c50 2023-11-17T00:09:55.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.797+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85101101f0 con 0x7f8510100c50 2023-11-17T00:09:55.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.800+0000 7f850d7fa700 1 -- 172.21.15.6:0/3380525252 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f8500020030 con 0x7f8510100c50 2023-11-17T00:09:55.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.801+0000 7f850d7fa700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 0x7f84fc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.801+0000 7f850d7fa700 1 -- 172.21.15.6:0/3380525252 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85000ce2b0 con 0x7f8510100c50 2023-11-17T00:09:55.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.801+0000 7f8507fff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 0x7f84fc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.802+0000 7f8507fff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 0x7f84fc0a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f85100745e0 tx=0x7f84f8009310 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.906+0000 7f8529be3b80 1 Processor -- start 2023-11-17T00:09:55.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 -- start start 2023-11-17T00:09:55.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d4d0060 con 0x55aa3d5cb470 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d5228a0 con 0x55aa3d5cab60 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8529be3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d51bd30 con 0x55aa3d5cf8b0 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.907+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54200/0 (socket says 172.21.15.6:54200) 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f8507fff700 1 -- 172.21.15.6:0/1033173435 learned_addr learned my addr 172.21.15.6:0/1033173435 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850ffff700 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 -- 172.21.15.6:0/1033173435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 msgr2=0x55aa3d5cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 -- 172.21.15.6:0/1033173435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 msgr2=0x55aa3d5cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 -- 172.21.15.6:0/1033173435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3d39d630 con 0x55aa3d5cab60 2023-11-17T00:09:55.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f8507fff700 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.908+0000 7f850f7fe700 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5caf30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85000cb4d0 tx=0x7f850000bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e994a43a95052a24 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8505ffb700 1 -- 172.21.15.6:0/1033173435 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850009ae10 con 0x55aa3d5cab60 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8505ffb700 1 -- 172.21.15.6:0/1033173435 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850009ee40 con 0x55aa3d5cab60 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8505ffb700 1 -- 172.21.15.6:0/1033173435 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500021eb0 con 0x55aa3d5cab60 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 -- 172.21.15.6:0/1033173435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 msgr2=0x55aa3d5caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5caf30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85000cb4d0 tx=0x7f850000bc90 comp rx=0 tx=0).stop 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 -- 172.21.15.6:0/1033173435 shutdown_connections 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5caf30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 --2- 172.21.15.6:0/1033173435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 -- 172.21.15.6:0/1033173435 >> 172.21.15.6:0/1033173435 conn(0x55aa3d5c0360 msgr2=0x55aa3d5c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.909+0000 7f8529be3b80 1 -- 172.21.15.6:0/1033173435 shutdown_connections 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.910+0000 7f8529be3b80 1 -- 172.21.15.6:0/1033173435 wait complete. 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.910+0000 7f8529be3b80 1 -- start start 2023-11-17T00:09:55.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.910+0000 7f8529be3b80 1 Processor -- start 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d4bd8e0 con 0x55aa3d5cab60 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d4bd580 con 0x55aa3d5cb470 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d39f920 con 0x55aa3d5cf8b0 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f8529be3b80 10 client.0 _ll_register_callbacks cb 0x55aa3d4db780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:55.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.911+0000 7f850f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f8507fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f8507fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42110/0 (socket says 172.21.15.6:42110) 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f8507fff700 1 -- 172.21.15.6:0/4229916274 learned_addr learned my addr 172.21.15.6:0/4229916274 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850ffff700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 -- 172.21.15.6:0/4229916274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 msgr2=0x55aa3d5d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 -- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 msgr2=0x55aa3d5d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 -- 172.21.15.6:0/4229916274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85101101f0 con 0x55aa3d5cab60 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f8507fff700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f850f7fe700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d6f70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85000ce220 tx=0x7f8500005170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7130c05caa892ff server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f84eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.912+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500021e50 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85000a31d0 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3d5bdb20 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85000cb0d0 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 10 client.15666 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55aa3d4e2190 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa3d5bdb20 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85000a2d40 con 0x55aa3d5cab60 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 20 client.15666 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d5e4010 0x55aa3d5e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 4 client.15666 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.913+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3d4de0a0 con 0x55aa3d5e4010 2023-11-17T00:09:55.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.914+0000 7f8507fff700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d5e4010 0x55aa3d5e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.914+0000 7f8507fff700 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d5e4010 0x55aa3d5e63f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53bfefd7ed3838f1 server_cookie=9eb3dd646a4adae2 in_seq=0 out_seq=0 2023-11-17T00:09:55.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.914+0000 7f84eeffd700 10 client.15666 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f84eeffd700 1 -- 172.21.15.6:0/4229916274 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa3d4de0a0 con 0x55aa3d5e4010 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f84eeffd700 10 client.15666 handle_command_reply: tid=0 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 client.15666 shutdown 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 msgr2=0x55aa3d5d6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d6f70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85000ce220 tx=0x7f8500005170 comp rx=0 tx=0).stop 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 shutdown_connections 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d5e4010 0x55aa3d5e63f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d6f70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f84eeffd700 0 client.15666 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:55.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 --2- 172.21.15.6:0/4229916274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 >> 172.21.15.6:0/4229916274 conn(0x55aa3d5c0360 msgr2=0x55aa3d5c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 shutdown_connections 2023-11-17T00:09:55.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.915+0000 7f8529be3b80 1 -- 172.21.15.6:0/4229916274 wait complete. 2023-11-17T00:09:55.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.916+0000 7f8529be3b80 20 client.15666 trim_cache size 0 max 16384 2023-11-17T00:09:55.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 Processor -- start 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 -- start start 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d5cbc10 con 0x55aa3d5cb470 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d5ce440 con 0x55aa3d5cf8b0 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.929+0000 7f8529be3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d5e69b0 con 0x55aa3d5cab60 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54226/0 (socket says 172.21.15.6:54226) 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 -- 172.21.15.6:0/2161486507 learned_addr learned my addr 172.21.15.6:0/2161486507 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:55.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f850ffff700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f850f7fe700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 -- 172.21.15.6:0/2161486507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cab60 msgr2=0x55aa3d5d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 -- 172.21.15.6:0/2161486507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 msgr2=0x55aa3d5ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 -- 172.21.15.6:0/2161486507 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3d4e1ee0 con 0x55aa3d5cb470 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f850ffff700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:55.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f8507fff700 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f84f8013010 tx=0x7f84f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabb7558a73fe795 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.930+0000 7f85067fc700 1 -- 172.21.15.6:0/2161486507 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8022040 con 0x55aa3d5cb470 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f85067fc700 1 -- 172.21.15.6:0/2161486507 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f8023040 con 0x55aa3d5cb470 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f85067fc700 1 -- 172.21.15.6:0/2161486507 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f801dcf0 con 0x55aa3d5cb470 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 -- 172.21.15.6:0/2161486507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 msgr2=0x55aa3d5d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f84f8013010 tx=0x7f84f8014000 comp rx=0 tx=0).stop 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 -- 172.21.15.6:0/2161486507 shutdown_connections 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d5ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cab60 0x55aa3d5d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2161486507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5d6aa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 -- 172.21.15.6:0/2161486507 >> 172.21.15.6:0/2161486507 conn(0x55aa3d5c0360 msgr2=0x55aa3d5e0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 -- 172.21.15.6:0/2161486507 shutdown_connections 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.931+0000 7f8529be3b80 1 -- 172.21.15.6:0/2161486507 wait complete. 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 -- start start 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 Processor -- start 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d636d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d5db760 con 0x55aa3d5cb470 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d6372a0 con 0x55aa3d5cab60 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d6373e0 con 0x55aa3d5cf8b0 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.932+0000 7f8529be3b80 10 client.0 _ll_register_callbacks cb 0x7f84f8009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:55.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54234/0 (socket says 172.21.15.6:54234) 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 -- 172.21.15.6:0/2069052040 learned_addr learned my addr 172.21.15.6:0/2069052040 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f850f7fe700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f850ffff700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d636d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 -- 172.21.15.6:0/2069052040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 msgr2=0x55aa3d636d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d636d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 -- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 msgr2=0x55aa3d5da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 -- 172.21.15.6:0/2069052040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f800e910 con 0x55aa3d5cb470 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f8507fff700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84f80095c0 tx=0x7f84f8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b218916a0e97ec server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f84eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.933+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8005240 con 0x55aa3d5cb470 2023-11-17T00:09:55.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f84f80053a0 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850009ee40 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f802b570 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 10 client.15672 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55aa3d332c60 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850009ee40 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84f803b070 con 0x55aa3d5cb470 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 20 client.15672 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d638a00 0x55aa3d63ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 4 client.15672 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.934+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa3d4ea660 con 0x55aa3d638a00 2023-11-17T00:09:55.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f850f7fe700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d638a00 0x55aa3d63ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f850f7fe700 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d638a00 0x55aa3d63ae10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ad3dd816bd3486 server_cookie=5c7b2ea9447d9ef5 in_seq=0 out_seq=0 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f84eeffd700 10 client.15672 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f84eeffd700 1 -- 172.21.15.6:0/2069052040 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa3d4ea660 con 0x55aa3d638a00 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f84eeffd700 10 client.15672 handle_command_reply: tid=0 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 client.15672 shutdown 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 msgr2=0x55aa3d5daf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84f80095c0 tx=0x7f84f8009740 comp rx=0 tx=0).stop 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 shutdown_connections 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa3d5cf8b0 0x55aa3d636d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55aa3d5cab60 0x55aa3d5da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55aa3d638a00 0x55aa3d63ae10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 --2- 172.21.15.6:0/2069052040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55aa3d5cb470 0x55aa3d5daf20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 >> 172.21.15.6:0/2069052040 conn(0x55aa3d5c0360 msgr2=0x55aa3d5d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.935+0000 7f84eeffd700 0 client.15672 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 shutdown_connections 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f8529be3b80 1 -- 172.21.15.6:0/2069052040 wait complete. 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f8529be3b80 20 client.15672 trim_cache size 0 max 16384 2023-11-17T00:09:55.941 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 msgr2=0x7f84fc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 0x7f84fc0a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f85100745e0 tx=0x7f84f8009310 comp rx=0 tx=0).stop 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 msgr2=0x7f8510072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 0x7f8510072df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8500009230 tx=0x7f850000fa20 comp rx=0 tx=0).stop 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 shutdown_connections 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84fc0a13e0 0x7f84fc0a3890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85101055c0 0x7f85101a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8510100c50 0x7f8510072df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 --2- 172.21.15.6:0/3380525252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85101015f0 0x7f8510073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 >> 172.21.15.6:0/3380525252 conn(0x7f8510077590 msgr2=0x7f85100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.936+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 shutdown_connections 2023-11-17T00:09:55.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:55.937+0000 7f85161ff700 1 -- 172.21.15.6:0/3380525252 wait complete. 2023-11-17T00:09:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:56.110 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:56.111 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:56.125 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:56.126 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:56.126 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 17 -lt 100 ']' 2023-11-17T00:09:56.126 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 17 + 1 2023-11-17T00:09:56.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=18 2023-11-17T00:09:56.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:56.187 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:56.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:56.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:56.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:56.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:56.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:56 smithi087 ceph-mon[131182]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-17T00:09:56.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:56 smithi186 ceph-mon[131487]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-17T00:09:56.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:56.871 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:56.871 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 18 -lt 100 ']' 2023-11-17T00:09:56.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 18 + 1 2023-11-17T00:09:56.899 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=19 2023-11-17T00:09:56.899 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:56.941 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:56.941 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:56.943 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:56.944 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:56.944 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:56.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:56 smithi006 ceph-mon[97840]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-17T00:09:56.993 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:09:56.993 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:09:57.448 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:57.448 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:57.565 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:57.566 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:57.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:57.584 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:57.584 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 19 -lt 100 ']' 2023-11-17T00:09:57.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 19 + 1 2023-11-17T00:09:57.621 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=20 2023-11-17T00:09:57.621 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:57.657 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:57.657 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:57.660 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:57.660 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:57.660 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:58.306 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:58.306 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:58.490 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:58.490 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:58.502 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:58.502 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:58.502 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 20 -lt 100 ']' 2023-11-17T00:09:58.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 20 + 1 2023-11-17T00:09:58.537 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=21 2023-11-17T00:09:58.538 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:58.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:58.599 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:58.601 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:58.602 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:58.602 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:58.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:09:58 smithi087 ceph-mon[131182]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-17T00:09:58.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:09:58 smithi186 ceph-mon[131487]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-17T00:09:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:09:58 smithi006 ceph-mon[97840]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-17T00:09:59.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:59.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:59.590 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:09:59.590 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:09:59.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 -- 172.21.15.6:0/3962294893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc10ce50 msgr2=0x7f89cc10f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 --2- 172.21.15.6:0/3962294893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc10ce50 0x7f89cc10f230 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b0a0 tx=0x7f89c400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 -- 172.21.15.6:0/3962294893 shutdown_connections 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 --2- 172.21.15.6:0/3962294893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc10ce50 0x7f89cc10f230 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 --2- 172.21.15.6:0/3962294893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc0748a0 0x7f89cc072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 --2- 172.21.15.6:0/3962294893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 0x7f89cc0736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 -- 172.21.15.6:0/3962294893 >> 172.21.15.6:0/3962294893 conn(0x7f89cc077530 msgr2=0x7f89cc077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 -- 172.21.15.6:0/3962294893 shutdown_connections 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.610+0000 7f89d0dff700 1 -- 172.21.15.6:0/3962294893 wait complete. 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 Processor -- start 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 -- start start 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 0x7f89cc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 0x7f89cc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc198d40 con 0x7f89cc0748a0 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc198eb0 con 0x7f89cc10ce50 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.611+0000 7f89d0dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc199070 con 0x7f89cc073260 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46164/0 (socket says 172.21.15.6:46164) 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 0x7f89cc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 -- 172.21.15.6:0/1505271114 learned_addr learned my addr 172.21.15.6:0/1505271114 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 0x7f89cc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 -- 172.21.15.6:0/1505271114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 msgr2=0x7f89cc19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 0x7f89cc19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 -- 172.21.15.6:0/1505271114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 msgr2=0x7f89cc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 0x7f89cc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89c3fff700 1 -- 172.21.15.6:0/1505271114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009d00 con 0x7f89cc0748a0 2023-11-17T00:09:59.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.612+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 0x7f89cc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e9a0 tx=0x7f89b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89c37fe700 1 -- 172.21.15.6:0/1505271114 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4009790 con 0x7f89cc0748a0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc112740 con 0x7f89cc0748a0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89c37fe700 1 -- 172.21.15.6:0/1505271114 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b400cbd0 con 0x7f89cc0748a0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89c37fe700 1 -- 172.21.15.6:0/1505271114 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4018680 con 0x7f89cc0748a0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.613+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc1125b0 con 0x7f89cc0748a0 2023-11-17T00:09:59.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.616+0000 7f89c37fe700 1 -- 172.21.15.6:0/1505271114 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f89b400cd40 con 0x7f89cc0748a0 2023-11-17T00:09:59.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.617+0000 7f89c37fe700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 0x7f89b80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.618+0000 7f89c37fe700 1 -- 172.21.15.6:0/1505271114 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f89b4014070 con 0x7f89cc0748a0 2023-11-17T00:09:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.618+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 0x7f89b80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:59.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:09:59.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 21 -lt 100 ']' 2023-11-17T00:09:59.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.618+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 0x7f89b80a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009be0 tx=0x7f89bc009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 21 + 1 2023-11-17T00:09:59.696 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=22 2023-11-17T00:09:59.696 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:09:59.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.697+0000 7f89e475cb80 1 Processor -- start 2023-11-17T00:09:59.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 -- start start 2023-11-17T00:09:59.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f254b0e0 con 0x5555f2645be0 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f259d920 con 0x5555f264a930 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89e475cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2596db0 con 0x5555f26464f0 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46166/0 (socket says 172.21.15.6:46166) 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89ca59c700 1 -- 172.21.15.6:0/1226242231 learned_addr learned my addr 172.21.15.6:0/1226242231 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.698+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 -- 172.21.15.6:0/1226242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 msgr2=0x5555f264a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 -- 172.21.15.6:0/1226242231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 msgr2=0x5555f264ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 -- 172.21.15.6:0/1226242231 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555f24186b0 con 0x5555f2645be0 2023-11-17T00:09:59.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:59.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:59.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f89bc011d70 tx=0x7f89bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b79a0ba8e6a4005 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89c0ff9700 1 -- 172.21.15.6:0/1226242231 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc011da0 con 0x5555f2645be0 2023-11-17T00:09:59.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89c0ff9700 1 -- 172.21.15.6:0/1226242231 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89bc004020 con 0x5555f2645be0 2023-11-17T00:09:59.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.699+0000 7f89c0ff9700 1 -- 172.21.15.6:0/1226242231 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc003bc0 con 0x5555f2645be0 2023-11-17T00:09:59.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 -- 172.21.15.6:0/1226242231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 msgr2=0x5555f2645fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f89bc011d70 tx=0x7f89bc014000 comp rx=0 tx=0).stop 2023-11-17T00:09:59.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 -- 172.21.15.6:0/1226242231 shutdown_connections 2023-11-17T00:09:59.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f2645fb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1226242231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 -- 172.21.15.6:0/1226242231 >> 172.21.15.6:0/1226242231 conn(0x5555f263b3e0 msgr2=0x5555f2640bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 -- 172.21.15.6:0/1226242231 shutdown_connections 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.700+0000 7f89e475cb80 1 -- 172.21.15.6:0/1226242231 wait complete. 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.701+0000 7f89e475cb80 1 -- start start 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.701+0000 7f89e475cb80 1 Processor -- start 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2538960 con 0x5555f2645be0 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2538600 con 0x5555f264a930 2023-11-17T00:09:59.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f241a9a0 con 0x5555f26464f0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89e475cb80 10 client.0 _ll_register_callbacks cb 0x5555f2556800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46176/0 (socket says 172.21.15.6:46176) 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 -- 172.21.15.6:0/3291449732 learned_addr learned my addr 172.21.15.6:0/3291449732 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89c3fff700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89cad9d700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 -- 172.21.15.6:0/3291449732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 msgr2=0x5555f264f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 -- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 msgr2=0x5555f264fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.702+0000 7f89ca59c700 1 -- 172.21.15.6:0/3291449732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc017070 con 0x5555f2645be0 2023-11-17T00:09:59.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89ca59c700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009210 tx=0x7f89bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e573b0b487bdb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc003a60 con 0x5555f2645be0 2023-11-17T00:09:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555f2638ba0 con 0x5555f2645be0 2023-11-17T00:09:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f89bc003bc0 con 0x5555f2645be0 2023-11-17T00:09:59.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89bc00ee70 con 0x5555f2645be0 2023-11-17T00:09:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89e475cb80 10 client.15684 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5555f255d210 con 0x5555f2645be0 2023-11-17T00:09:59.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.703+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555f2638ba0 con 0x5555f2645be0 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89bc024030 con 0x5555f2645be0 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 20 client.15684 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f265f850 0x5555f2661c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 4 client.15684 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:09:59.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555f2559120 con 0x5555f265f850 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.704+0000 7f89c3fff700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f265f850 0x5555f2661c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.705+0000 7f89c3fff700 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f265f850 0x5555f2661c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56cb1089002a423a server_cookie=1a3f615f05261f09 in_seq=0 out_seq=0 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.705+0000 7f89b1ffb700 10 client.15684 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.705+0000 7f89b1ffb700 1 -- 172.21.15.6:0/3291449732 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5555f2559120 con 0x5555f265f850 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89b1ffb700 10 client.15684 handle_command_reply: tid=0 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 client.15684 shutdown 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 msgr2=0x5555f26520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009210 tx=0x7f89bc011ad0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 shutdown_connections 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f265f850 0x5555f2661c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f2645be0 0x5555f26520f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 --2- 172.21.15.6:0/3291449732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f26464f0 0x5555f264f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 >> 172.21.15.6:0/3291449732 conn(0x5555f263b3e0 msgr2=0x5555f264bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89b1ffb700 0 client.15684 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 shutdown_connections 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 1 -- 172.21.15.6:0/3291449732 wait complete. 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.706+0000 7f89e475cb80 20 client.15684 trim_cache size 0 max 16384 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.724+0000 7f89e475cb80 1 Processor -- start 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.724+0000 7f89e475cb80 1 -- start start 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.724+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f2648220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89e475cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2650390 con 0x5555f26464f0 2023-11-17T00:09:59.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89e475cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2641930 con 0x5555f264a930 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89e475cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f2638790 con 0x5555f2645be0 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46182/0 (socket says 172.21.15.6:46182) 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 -- 172.21.15.6:0/1292916875 learned_addr learned my addr 172.21.15.6:0/1292916875 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89ca59c700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f2648220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 -- 172.21.15.6:0/1292916875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 msgr2=0x5555f2648220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f2648220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 -- 172.21.15.6:0/1292916875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 msgr2=0x5555f264a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 -- 172.21.15.6:0/1292916875 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555f263cb80 con 0x5555f26464f0 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89cad9d700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.725+0000 7f89c3fff700 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89b4096b10 tx=0x7f89b401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fe5c13daa54161 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89c17fa700 1 -- 172.21.15.6:0/1292916875 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40cd040 con 0x5555f26464f0 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89c17fa700 1 -- 172.21.15.6:0/1292916875 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b40a24f0 con 0x5555f26464f0 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89c17fa700 1 -- 172.21.15.6:0/1292916875 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40997f0 con 0x5555f26464f0 2023-11-17T00:09:59.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 -- 172.21.15.6:0/1292916875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 msgr2=0x5555f2656a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89b4096b10 tx=0x7f89b401a040 comp rx=0 tx=0).stop 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 -- 172.21.15.6:0/1292916875 shutdown_connections 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f264a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f2648220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 --2- 172.21.15.6:0/1292916875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f2656a30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 -- 172.21.15.6:0/1292916875 >> 172.21.15.6:0/1292916875 conn(0x5555f263b3e0 msgr2=0x5555f26499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 -- 172.21.15.6:0/1292916875 shutdown_connections 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.726+0000 7f89e475cb80 1 -- 172.21.15.6:0/1292916875 wait complete. 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.727+0000 7f89e475cb80 1 -- start start 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.727+0000 7f89e475cb80 1 Processor -- start 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.727+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f26ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.727+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.727+0000 7f89e475cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f26a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89e475cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f26a8040 con 0x5555f26464f0 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89e475cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f26a81b0 con 0x5555f264a930 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89e475cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f26a8370 con 0x5555f2645be0 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89e475cb80 10 client.0 _ll_register_callbacks cb 0x7f89b4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89ca59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f26ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46194/0 (socket says 172.21.15.6:46194) 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 -- 172.21.15.6:0/2659838020 learned_addr learned my addr 172.21.15.6:0/2659838020 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89cad9d700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f26a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 -- 172.21.15.6:0/2659838020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 msgr2=0x5555f26ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f26ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 -- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 msgr2=0x5555f26a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f26a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 -- 172.21.15.6:0/2659838020 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc112740 con 0x5555f26464f0 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89ca59c700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f26ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89cad9d700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f26a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89c3fff700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89b40a1fc0 tx=0x7f89b409e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400faca689e263f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:09:59.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.728+0000 7f89b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40cd040 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89bc004020 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f89b40b82b0 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b40b8470 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 10 client.15690 fetch_fsmap learned FSMap version 20 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5555f23c4e60 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b4063080 con 0x5555f26464f0 2023-11-17T00:09:59.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89b40a2060 con 0x5555f26464f0 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 20 client.15690 populate_metadata read hostname 'smithi006' 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f26b5290 0x5555f26b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 4 client.15690 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.729+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5555f25656e0 con 0x5555f26b5290 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.730+0000 7f89ca59c700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f26b5290 0x5555f26b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.730+0000 7f89ca59c700 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f26b5290 0x5555f26b76a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c838e77cfef580f server_cookie=d43f67034c0dfbb7 in_seq=0 out_seq=0 2023-11-17T00:09:59.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.730+0000 7f89b1ffb700 10 client.15690 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89b1ffb700 1 -- 172.21.15.6:0/2659838020 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5555f25656e0 con 0x5555f26b5290 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89b1ffb700 10 client.15690 handle_command_reply: tid=0 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 client.15690 shutdown 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 msgr2=0x5555f26adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89b40a1fc0 tx=0x7f89b409e250 comp rx=0 tx=0).stop 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 shutdown_connections 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5555f264a930 0x5555f26a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5555f2645be0 0x5555f26ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5555f26b5290 0x5555f26b76a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 --2- 172.21.15.6:0/2659838020 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5555f26464f0 0x5555f26adc00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 >> 172.21.15.6:0/2659838020 conn(0x5555f263b3e0 msgr2=0x5555f26449e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89b1ffb700 0 client.15690 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:09:59.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 shutdown_connections 2023-11-17T00:09:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.733+0000 7f89e475cb80 1 -- 172.21.15.6:0/2659838020 wait complete. 2023-11-17T00:09:59.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89e475cb80 20 client.15690 trim_cache size 0 max 16384 2023-11-17T00:09:59.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:09:59.805 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "22e505e3-02b6-490c-a4ed-50b37ee9f24f", 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 msgr2=0x7f89b80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 0x7f89b80a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009be0 tx=0x7f89bc009260 comp rx=0 tx=0).stop 2023-11-17T00:09:59.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 msgr2=0x7f89cc19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:09:59.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e9a0 tx=0x7f89b400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.807 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:09:59.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 shutdown_connections 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f89b80a13e0 0x7f89b80a3890 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009be0 tx=0x7f89bc009260 comp rx=0 tx=0).stop 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc10ce50 0x7f89cc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89cc073260 0x7f89cc19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 --2- 172.21.15.6:0/1505271114 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89cc0748a0 0x7f89cc19eba0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.734+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 >> 172.21.15.6:0/1505271114 conn(0x7f89cc077530 msgr2=0x7f89cc104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.735+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 shutdown_connections 2023-11-17T00:09:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:09:59.735+0000 7f89d0dff700 1 -- 172.21.15.6:0/1505271114 wait complete. 2023-11-17T00:09:59.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:09:59.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:09:59.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:09:59.810 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:10:00.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:00.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:00.424 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:00.425 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:00.479 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:00.479 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:00.479 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 22 -lt 100 ']' 2023-11-17T00:10:00.479 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 22 + 1 2023-11-17T00:10:00.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=23 2023-11-17T00:10:00.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:00.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:00.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:00.682 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:00.683 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:00.683 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:00.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: overall HEALTH_OK 2023-11-17T00:10:00.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:00 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:10:00.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-11-17T00:10:00.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:10:00.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:10:00.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: overall HEALTH_OK 2023-11-17T00:10:00.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:00 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:10:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-11-17T00:10:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:10:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:10:00.908 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:00.908 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: overall HEALTH_OK 2023-11-17T00:10:00.909 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:00 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:10:01.248 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:01.248 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:01.466 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:01.467 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:01.467 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 23 -lt 100 ']' 2023-11-17T00:10:01.467 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 23 + 1 2023-11-17T00:10:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=24 2023-11-17T00:10:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:01.590 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:01.590 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:01.592 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:02.272 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:02.272 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:02.501 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:02.501 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:02.526 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:02.527 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:02.527 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 24 -lt 100 ']' 2023-11-17T00:10:02.527 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 24 + 1 2023-11-17T00:10:02.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.534+0000 7f872b1ff700 1 -- 172.21.15.6:0/2210725213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724067dc0 msgr2=0x7f8724068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.534+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2210725213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724067dc0 0x7f8724068230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8710009990 tx=0x7f8710009ca0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 -- 172.21.15.6:0/2210725213 shutdown_connections 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2210725213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f872410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2210725213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724105250 0x7f8724105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2210725213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724067dc0 0x7f8724068230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 -- 172.21.15.6:0/2210725213 >> 172.21.15.6:0/2210725213 conn(0x7f87240763a0 msgr2=0x7f87240767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 -- 172.21.15.6:0/2210725213 shutdown_connections 2023-11-17T00:10:02.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.535+0000 7f872b1ff700 1 -- 172.21.15.6:0/2210725213 wait complete. 2023-11-17T00:10:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 Processor -- start 2023-11-17T00:10:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 -- start start 2023-11-17T00:10:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724067dc0 0x7f8724198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 0x7f872419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419d640 con 0x7f8724068770 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419d7b0 con 0x7f8724105250 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.536+0000 7f872b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419d970 con 0x7f8724067dc0 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46212/0 (socket says 172.21.15.6:46212) 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8728f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724067dc0 0x7f8724198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 -- 172.21.15.6:0/2834499164 learned_addr learned my addr 172.21.15.6:0/2834499164 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f872979c700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 0x7f872419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 -- 172.21.15.6:0/2834499164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724067dc0 msgr2=0x7f8724198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724067dc0 0x7f8724198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 -- 172.21.15.6:0/2834499164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 msgr2=0x7f872419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 0x7f872419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 -- 172.21.15.6:0/2834499164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100095f0 con 0x7f8724068770 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f872979c700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 0x7f872419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8723fff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8710005910 tx=0x7f8710004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.537+0000 7f8721ffb700 1 -- 172.21.15.6:0/2834499164 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871001d070 con 0x7f8724068770 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.538+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8724110420 con 0x7f8724068770 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.538+0000 7f8721ffb700 1 -- 172.21.15.6:0/2834499164 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8710022470 con 0x7f8724068770 2023-11-17T00:10:02.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.538+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724110290 con 0x7f8724068770 2023-11-17T00:10:02.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.538+0000 7f8721ffb700 1 -- 172.21.15.6:0/2834499164 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871002c640 con 0x7f8724068770 2023-11-17T00:10:02.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.541+0000 7f8721ffb700 1 -- 172.21.15.6:0/2834499164 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f87100225e0 con 0x7f8724068770 2023-11-17T00:10:02.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.542+0000 7f8721ffb700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 0x7f87140a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.542+0000 7f8721ffb700 1 -- 172.21.15.6:0/2834499164 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87100ce610 con 0x7f8724068770 2023-11-17T00:10:02.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.542+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 0x7f87140a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.543+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 0x7f87140a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8718009c60 tx=0x7f8718009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=25 2023-11-17T00:10:02.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:02.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 Processor -- start 2023-11-17T00:10:02.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 -- start start 2023-11-17T00:10:02.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c09d060 con 0x55ca6c198470 2023-11-17T00:10:02.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c0ef8a0 con 0x55ca6c197b60 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.622+0000 7f873eaccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c0e8d30 con 0x55ca6c19c8b0 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46220/0 (socket says 172.21.15.6:46220) 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 -- 172.21.15.6:0/3908946549 learned_addr learned my addr 172.21.15.6:0/3908946549 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f872979c700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8728f9b700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 -- 172.21.15.6:0/3908946549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 msgr2=0x55ca6c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 -- 172.21.15.6:0/3908946549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 msgr2=0x55ca6c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 -- 172.21.15.6:0/3908946549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca6bf6a630 con 0x55ca6c198470 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8728f9b700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.623+0000 7f8723fff700 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f871009ae70 tx=0x7f8710096ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c976759f5a71fdbf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.624+0000 7f870effd700 1 -- 172.21.15.6:0/3908946549 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871009aad0 con 0x55ca6c198470 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.624+0000 7f870effd700 1 -- 172.21.15.6:0/3908946549 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871009b170 con 0x55ca6c198470 2023-11-17T00:10:02.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.624+0000 7f870effd700 1 -- 172.21.15.6:0/3908946549 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87100ce440 con 0x55ca6c198470 2023-11-17T00:10:02.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.624+0000 7f873eaccb80 1 -- 172.21.15.6:0/3908946549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 msgr2=0x55ca6c19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.624+0000 7f873eaccb80 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f871009ae70 tx=0x7f8710096ea0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 -- 172.21.15.6:0/3908946549 shutdown_connections 2023-11-17T00:10:02.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 --2- 172.21.15.6:0/3908946549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c19c370 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 -- 172.21.15.6:0/3908946549 >> 172.21.15.6:0/3908946549 conn(0x55ca6c18d360 msgr2=0x55ca6c192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 -- 172.21.15.6:0/3908946549 shutdown_connections 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 -- 172.21.15.6:0/3908946549 wait complete. 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.625+0000 7f873eaccb80 1 -- start start 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 Processor -- start 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c1a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c08a8e0 con 0x55ca6c198470 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c08a580 con 0x55ca6c197b60 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6bf6c920 con 0x55ca6c19c8b0 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f873eaccb80 10 client.0 _ll_register_callbacks cb 0x55ca6c0a8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f8728f9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c1a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46236/0 (socket says 172.21.15.6:46236) 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.626+0000 7f8723fff700 1 -- 172.21.15.6:0/2550803607 learned_addr learned my addr 172.21.15.6:0/2550803607 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f872979c700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 -- 172.21.15.6:0/2550803607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 msgr2=0x55ca6c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 -- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 msgr2=0x55ca6c1a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c1a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 -- 172.21.15.6:0/2550803607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724110290 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8723fff700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8710024000 tx=0x7f871009ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959616668adb521c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8703fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871009f1a0 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f871009a870 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca6c18ab20 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87100a30a0 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f873eaccb80 10 client.15702 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.627+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ca6c0af190 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca6c18ab20 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87100cbb10 con 0x55ca6c198470 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 10 client.15702 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:02.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 20 client.15702 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c1b1070 0x55ca6c1b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 4 client.15702 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca6c0ab0a0 con 0x55ca6c1b1070 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.628+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c1b1070 0x55ca6c1b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.629+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c1b1070 0x55ca6c1b3450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3069c5951eaf1107 server_cookie=fa6baab6fb7ed0e7 in_seq=0 out_seq=0 2023-11-17T00:10:02.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.629+0000 7f8703fff700 10 client.15702 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:02.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f8703fff700 1 -- 172.21.15.6:0/2550803607 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca6c0ab0a0 con 0x55ca6c1b1070 2023-11-17T00:10:02.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f8703fff700 10 client.15702 handle_command_reply: tid=0 2023-11-17T00:10:02.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 client.15702 shutdown 2023-11-17T00:10:02.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 msgr2=0x55ca6c1a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8710024000 tx=0x7f871009ad70 comp rx=0 tx=0).stop 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 shutdown_connections 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c197b60 0x55ca6c1a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c1b1070 0x55ca6c1b3450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2550803607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1a11a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 >> 172.21.15.6:0/2550803607 conn(0x55ca6c18d360 msgr2=0x55ca6c19dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f8703fff700 0 client.15702 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 shutdown_connections 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 1 -- 172.21.15.6:0/2550803607 wait complete. 2023-11-17T00:10:02.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.630+0000 7f873eaccb80 20 client.15702 trim_cache size 0 max 16384 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 Processor -- start 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 -- start start 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c198470 0x55ca6c192c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c0ab0a0 con 0x55ca6c19c8b0 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c198bc0 con 0x55ca6c198470 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.648+0000 7f873eaccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c1938b0 con 0x55ca6c197b60 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f872979c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f872979c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46238/0 (socket says 172.21.15.6:46238) 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53662/0 (socket says 172.21.15.6:53662) 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f872979c700 1 -- 172.21.15.6:0/2221036385 learned_addr learned my addr 172.21.15.6:0/2221036385 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8723fff700 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c198470 0x55ca6c192c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 -- 172.21.15.6:0/2221036385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c198470 msgr2=0x55ca6c192c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c198470 0x55ca6c192c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 -- 172.21.15.6:0/2221036385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 msgr2=0x55ca6c1a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 -- 172.21.15.6:0/2221036385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca6c0aeee0 con 0x55ca6c197b60 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f872979c700 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.649+0000 7f8728f9b700 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f871800a010 tx=0x7f8718014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f00291beb48d0e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f870f7fe700 1 -- 172.21.15.6:0/2221036385 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718023040 con 0x55ca6c197b60 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f870f7fe700 1 -- 172.21.15.6:0/2221036385 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8718028c30 con 0x55ca6c197b60 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f870f7fe700 1 -- 172.21.15.6:0/2221036385 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871801d670 con 0x55ca6c197b60 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 -- 172.21.15.6:0/2221036385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 msgr2=0x55ca6c1a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f871800a010 tx=0x7f8718014000 comp rx=0 tx=0).stop 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 -- 172.21.15.6:0/2221036385 shutdown_connections 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1a0450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 --2- 172.21.15.6:0/2221036385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c198470 0x55ca6c192c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 -- 172.21.15.6:0/2221036385 >> 172.21.15.6:0/2221036385 conn(0x55ca6c18d360 msgr2=0x55ca6c1ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 -- 172.21.15.6:0/2221036385 shutdown_connections 2023-11-17T00:10:02.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.650+0000 7f873eaccb80 1 -- 172.21.15.6:0/2221036385 wait complete. 2023-11-17T00:10:02.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.651+0000 7f873eaccb80 1 -- start start 2023-11-17T00:10:02.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.651+0000 7f873eaccb80 1 Processor -- start 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.651+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c1a4cb0 con 0x55ca6c198470 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c1a4e20 con 0x55ca6c19c8b0 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6c1a4fe0 con 0x55ca6c197b60 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f873eaccb80 10 client.0 _ll_register_callbacks cb 0x55ca6c0a8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8728f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46252/0 (socket says 172.21.15.6:46252) 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 -- 172.21.15.6:0/1586482851 learned_addr learned my addr 172.21.15.6:0/1586482851 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:02.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f872979c700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 -- 172.21.15.6:0/1586482851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 msgr2=0x55ca6c1fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 -- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c19c8b0 msgr2=0x55ca6c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.652+0000 7f8723fff700 1 -- 172.21.15.6:0/1586482851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c009260 con 0x55ca6c198470 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8723fff700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8710022c80 tx=0x7f871000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca05cf615edc43e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871009aad0 con 0x55ca6c198470 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f871009f1a0 con 0x55ca6c198470 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f871009b170 con 0x55ca6c198470 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f871009a4a0 con 0x55ca6c198470 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f873eaccb80 10 client.15708 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:02.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8718017070 con 0x55ca6c198470 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871009b170 con 0x55ca6c198470 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.653+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87100970b0 con 0x55ca6c198470 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 20 client.15708 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c205c50 0x55ca6c208060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 4 client.15708 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca6c0b7660 con 0x55ca6c205c50 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.654+0000 7f8728f9b700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c205c50 0x55ca6c208060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f8703fff700 10 client.15708 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f8728f9b700 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c205c50 0x55ca6c208060 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8c024ddbaa68b5 server_cookie=15f1f7eda9d0ccf6 in_seq=0 out_seq=0 2023-11-17T00:10:02.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f8703fff700 1 -- 172.21.15.6:0/1586482851 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ca6c0b7660 con 0x55ca6c205c50 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f8703fff700 10 client.15708 handle_command_reply: tid=0 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 client.15708 shutdown 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 msgr2=0x55ca6c1fd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8710022c80 tx=0x7f871000b690 comp rx=0 tx=0).stop 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 shutdown_connections 2023-11-17T00:10:02.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca6c19c8b0 0x55ca6c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca6c197b60 0x55ca6c1fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca6c205c50 0x55ca6c208060 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 --2- 172.21.15.6:0/1586482851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca6c198470 0x55ca6c1fd6c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 >> 172.21.15.6:0/1586482851 conn(0x55ca6c18d360 msgr2=0x55ca6c1a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f8703fff700 0 client.15708 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 shutdown_connections 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 1 -- 172.21.15.6:0/1586482851 wait complete. 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.655+0000 7f873eaccb80 20 client.15708 trim_cache size 0 max 16384 2023-11-17T00:10:02.665 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:10:02.666 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:10:02.666 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 msgr2=0x7f87140a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 0x7f87140a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8718009c60 tx=0x7f8718009260 comp rx=0 tx=0).stop 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 msgr2=0x7f8724198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8710005910 tx=0x7f8710004710 comp rx=0 tx=0).stop 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 shutdown_connections 2023-11-17T00:10:02.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87140a13e0 0x7f87140a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8724105250 0x7f872419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8724067dc0 0x7f8724198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 --2- 172.21.15.6:0/2834499164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8724068770 0x7f8724198ba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:02.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 >> 172.21.15.6:0/2834499164 conn(0x7f87240763a0 msgr2=0x7f872410c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:02.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:02.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 shutdown_connections 2023-11-17T00:10:02.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:02.656+0000 7f872b1ff700 1 -- 172.21.15.6:0/2834499164 wait complete. 2023-11-17T00:10:02.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:02.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:02.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:02.681 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:02.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:10:02.709 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:10:02.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:02 smithi087 ceph-mon[131182]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:02.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:02 smithi186 ceph-mon[131487]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:02.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:02 smithi006 ceph-mon[97840]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:03.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:03.533 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:03.617 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:03.618 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:03.664 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:03.665 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 25 -lt 100 ']' 2023-11-17T00:10:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 25 + 1 2023-11-17T00:10:03.695 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=26 2023-11-17T00:10:03.695 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:03.713 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:03.713 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:03.714 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:03.715 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:03.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:03.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:03 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:10:03.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:03 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:10:03.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:03 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:10:04.263 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:04.264 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:04.390 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:04.390 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:04.425 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:04.425 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:04.425 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 26 -lt 100 ']' 2023-11-17T00:10:04.425 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 26 + 1 2023-11-17T00:10:04.465 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=27 2023-11-17T00:10:04.465 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:04.573 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:04.574 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:04.581 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:04.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:04.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:04 smithi087 ceph-mon[131182]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:04 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:04 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:04 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:04 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:04 smithi186 ceph-mon[131487]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:04 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:04 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:04 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:04 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:04 smithi006 ceph-mon[97840]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:10:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:04 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:04 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:04 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:04 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:05.405 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:05.405 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:05.634 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:05.634 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:05.668 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:05.669 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:05.669 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 27 -lt 100 ']' 2023-11-17T00:10:05.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 27 + 1 2023-11-17T00:10:05.820 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=28 2023-11-17T00:10:05.820 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 -- 172.21.15.6:0/3033243619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 msgr2=0x7faf6c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 --2- 172.21.15.6:0/3033243619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c10f2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faf64009990 tx=0x7faf64009ca0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 -- 172.21.15.6:0/713225873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 msgr2=0x7f60e8101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/713225873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8101090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098f0 tx=0x7f60d8009c00 comp rx=0 tx=0).stop 2023-11-17T00:10:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 -- 172.21.15.6:0/3033243619 shutdown_connections 2023-11-17T00:10:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 --2- 172.21.15.6:0/3033243619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c10f2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 --2- 172.21.15.6:0/3033243619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c108490 0x7faf6c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 --2- 172.21.15.6:0/3033243619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c102480 0x7faf6c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 -- 172.21.15.6:0/3033243619 >> 172.21.15.6:0/3033243619 conn(0x7faf6c0fdf60 msgr2=0x7faf6c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 -- 172.21.15.6:0/713225873 shutdown_connections 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/713225873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 0x7f60e810d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/713225873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8101090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/713225873 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 -- 172.21.15.6:0/713225873 >> 172.21.15.6:0/713225873 conn(0x7f60e80774d0 msgr2=0x7f60e80778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:05.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.893+0000 7f60ed5ff700 1 -- 172.21.15.6:0/713225873 shutdown_connections 2023-11-17T00:10:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 -- 172.21.15.6:0/3033243619 shutdown_connections 2023-11-17T00:10:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.893+0000 7faf709ff700 1 -- 172.21.15.6:0/3033243619 wait complete. 2023-11-17T00:10:05.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.894+0000 7f60ed5ff700 1 -- 172.21.15.6:0/713225873 wait complete. 2023-11-17T00:10:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.894+0000 7faf709ff700 1 Processor -- start 2023-11-17T00:10:05.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.894+0000 7f60ed5ff700 1 Processor -- start 2023-11-17T00:10:05.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 -- start start 2023-11-17T00:10:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 -- start start 2023-11-17T00:10:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8194300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19d550 con 0x7faf6c108490 2023-11-17T00:10:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19d6c0 con 0x7faf6c102e30 2023-11-17T00:10:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.895+0000 7faf709ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19d880 con 0x7faf6c102480 2023-11-17T00:10:05.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 0x7f60e8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81992e0 con 0x7f60e8101660 2023-11-17T00:10:05.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8199450 con 0x7f60e8105620 2023-11-17T00:10:05.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.895+0000 7f60ed5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8199610 con 0x7f60e8100cc0 2023-11-17T00:10:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53690/0 (socket says 172.21.15.6:53690) 2023-11-17T00:10:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46270/0 (socket says 172.21.15.6:46270) 2023-11-17T00:10:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6a59c700 1 -- 172.21.15.6:0/1018917401 learned_addr learned my addr 172.21.15.6:0/1018917401 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:05.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46272/0 (socket says 172.21.15.6:46272) 2023-11-17T00:10:05.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 0x7f60e8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 -- 172.21.15.6:0/2265140430 learned_addr learned my addr 172.21.15.6:0/2265140430 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e6ffd700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8194300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 -- 172.21.15.6:0/1018917401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 msgr2=0x7faf6c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 -- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 msgr2=0x7faf6c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 -- 172.21.15.6:0/2265140430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 msgr2=0x7f60e8194300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8194300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 -- 172.21.15.6:0/2265140430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 msgr2=0x7f60e8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:05.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 0x7f60e8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:05.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.896+0000 7f60e67fc700 1 -- 172.21.15.6:0/2265140430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009520 con 0x7f60e8101660 2023-11-17T00:10:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf69d9b700 1 -- 172.21.15.6:0/1018917401 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640095f0 con 0x7faf6c102e30 2023-11-17T00:10:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.896+0000 7faf6a59c700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf69d9b700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c198ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9f0 tx=0x7faf6000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60e67fc700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60dc012f90 tx=0x7f60dc00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc01c070 con 0x7f60e8101660 2023-11-17T00:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf5b7fe700 1 -- 172.21.15.6:0/1018917401 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000c9c0 con 0x7faf6c102e30 2023-11-17T00:10:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e810e230 con 0x7f60e8101660 2023-11-17T00:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf5b7fe700 1 -- 172.21.15.6:0/1018917401 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf60010430 con 0x7faf6c102e30 2023-11-17T00:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c1124d0 con 0x7faf6c102e30 2023-11-17T00:10:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf5b7fe700 1 -- 172.21.15.6:0/1018917401 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60018680 con 0x7faf6c102e30 2023-11-17T00:10:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60dc013040 con 0x7f60e8101660 2023-11-17T00:10:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc005520 con 0x7f60e8101660 2023-11-17T00:10:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.897+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c112340 con 0x7faf6c102e30 2023-11-17T00:10:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.897+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e810e0a0 con 0x7f60e8101660 2023-11-17T00:10:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.899+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e810e3c0 con 0x7f60e8101660 2023-11-17T00:10:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.900+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f60dc004750 con 0x7f60e8101660 2023-11-17T00:10:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.901+0000 7faf5b7fe700 1 -- 172.21.15.6:0/1018917401 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7faf60010860 con 0x7faf6c102e30 2023-11-17T00:10:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.903+0000 7f60cffff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 0x7f60d00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.903+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60dc0d0cb0 con 0x7f60e8101660 2023-11-17T00:10:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.903+0000 7f60e6ffd700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 0x7f60d00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.903+0000 7faf5b7fe700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 0x7faf540a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.903+0000 7faf5b7fe700 1 -- 172.21.15.6:0/1018917401 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faf60014070 con 0x7faf6c102e30 2023-11-17T00:10:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.904+0000 7faf6a59c700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 0x7faf540a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:05.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:05.904+0000 7f60e6ffd700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 0x7f60d00a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60d8005e90 tx=0x7f60d8019040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:05.904+0000 7faf6a59c700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 0x7faf540a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009c10 tx=0x7faf5c00b040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:05.944 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:05.944 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:05.944 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:05.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:05.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:06.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.157+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f60dc099ac0 con 0x7f60e8101660 2023-11-17T00:10:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 Processor -- start 2023-11-17T00:10:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 -- start start 2023-11-17T00:10:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 0x563bb52421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb523c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb523cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb5142000 con 0x563bb5242720 2023-11-17T00:10:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb5194840 con 0x563bb50986b0 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.195+0000 7faf8432bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb518dcd0 con 0x563bb5242b90 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 0x563bb52421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb523cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 0x563bb52421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:51348/0 (socket says 172.21.15.6:51348) 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6a59c700 1 -- 172.21.15.6:0/2007271833 learned_addr learned my addr 172.21.15.6:0/2007271833 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf69d9b700 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb523c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 -- 172.21.15.6:0/2007271833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 msgr2=0x563bb52421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 0x563bb52421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 -- 172.21.15.6:0/2007271833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 msgr2=0x563bb523c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb523c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 -- 172.21.15.6:0/2007271833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb500f5d0 con 0x563bb5242b90 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.196+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb523cb00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf640098e0 tx=0x7faf64005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0ac341070b3e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.197+0000 7faf58ff9700 1 -- 172.21.15.6:0/2007271833 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6401d070 con 0x563bb5242b90 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.197+0000 7faf58ff9700 1 -- 172.21.15.6:0/2007271833 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6401e040 con 0x563bb5242b90 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.197+0000 7faf58ff9700 1 -- 172.21.15.6:0/2007271833 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64004c80 con 0x563bb5242b90 2023-11-17T00:10:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.197+0000 7faf8432bb80 1 -- 172.21.15.6:0/2007271833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 msgr2=0x563bb523cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.197+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb523cb00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf640098e0 tx=0x7faf64005ee0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 -- 172.21.15.6:0/2007271833 shutdown_connections 2023-11-17T00:10:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb523cb00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb50986b0 0x563bb52421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2007271833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb523c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 -- 172.21.15.6:0/2007271833 >> 172.21.15.6:0/2007271833 conn(0x563bb5231f50 msgr2=0x563bb5237270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 -- 172.21.15.6:0/2007271833 shutdown_connections 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 -- 172.21.15.6:0/2007271833 wait complete. 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 -- start start 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.198+0000 7faf8432bb80 1 Processor -- start 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb524ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb524bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5245d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb512f880 con 0x563bb50986b0 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb512f520 con 0x563bb5242720 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb50118c0 con 0x563bb5242b90 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf8432bb80 10 client.0 _ll_register_callbacks cb 0x563bb514d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb524ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb524ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46318/0 (socket says 172.21.15.6:46318) 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf69d9b700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb524bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.199+0000 7faf6a59c700 1 -- 172.21.15.6:0/3330589756 learned_addr learned my addr 172.21.15.6:0/3330589756 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5245d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 -- 172.21.15.6:0/3330589756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 msgr2=0x563bb5245d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5245d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 -- 172.21.15.6:0/3330589756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 msgr2=0x563bb524ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb524ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 -- 172.21.15.6:0/3330589756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c10fbc0 con 0x563bb5242720 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf69d9b700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb524bfc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf600ca660 tx=0x7faf6009a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa42be595eb58e35 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600b1070 con 0x563bb5242720 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faf60099f10 con 0x563bb5242720 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.200+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bb522f710 con 0x563bb5242720 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf6009df40 con 0x563bb5242720 2023-11-17T00:10:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 10 client.25790 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563bb5154130 con 0x563bb5242720 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bb522f710 con 0x563bb5242720 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf600a1ed0 con 0x563bb5242720 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 20 client.25790 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb5255ed0 0x563bb52582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 4 client.25790 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bb5150040 con 0x563bb5255ed0 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.201+0000 7faf6a59c700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb5255ed0 0x563bb52582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.202+0000 7faf6a59c700 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb5255ed0 0x563bb52582e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f570aa9f99e6487 server_cookie=e8c1dea8fcfe98bd in_seq=0 out_seq=0 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.202+0000 7faf51ffb700 10 client.25790 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf51ffb700 1 -- 172.21.15.6:0/3330589756 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563bb5150040 con 0x563bb5255ed0 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf51ffb700 10 client.25790 handle_command_reply: tid=0 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 client.25790 shutdown 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 msgr2=0x563bb524bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb524bfc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf600ca660 tx=0x7faf6009a450 comp rx=0 tx=0).stop 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 shutdown_connections 2023-11-17T00:10:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb5255ed0 0x563bb52582e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb524ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5245d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 --2- 172.21.15.6:0/3330589756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb524bfc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf51ffb700 0 client.25790 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 >> 172.21.15.6:0/3330589756 conn(0x563bb5231f50 msgr2=0x563bb52367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 shutdown_connections 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 1 -- 172.21.15.6:0/3330589756 wait complete. 2023-11-17T00:10:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.203+0000 7faf8432bb80 20 client.25790 trim_cache size 0 max 16384 2023-11-17T00:10:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 Processor -- start 2023-11-17T00:10:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 -- start start 2023-11-17T00:10:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb52414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5236f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb523d250 con 0x563bb50986b0 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb5244290 con 0x563bb5242720 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf8432bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb52409c0 con 0x563bb5242b90 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46326/0 (socket says 172.21.15.6:46326) 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf69d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb52414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.220+0000 7faf6a59c700 1 -- 172.21.15.6:0/2369496083 learned_addr learned my addr 172.21.15.6:0/2369496083 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5236f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 -- 172.21.15.6:0/2369496083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 msgr2=0x563bb5236f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5236f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 -- 172.21.15.6:0/2369496083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 msgr2=0x563bb5245e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 -- 172.21.15.6:0/2369496083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb525cf10 con 0x563bb5242720 2023-11-17T00:10:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf6a59c700 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf69d9b700 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb52414e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf60096980 tx=0x7faf60010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad232ada8b1af172 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf597fa700 1 -- 172.21.15.6:0/2369496083 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600b1070 con 0x563bb5242720 2023-11-17T00:10:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.221+0000 7faf597fa700 1 -- 172.21.15.6:0/2369496083 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf600ca990 con 0x563bb5242720 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf597fa700 1 -- 172.21.15.6:0/2369496083 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600b81a0 con 0x563bb5242720 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 -- 172.21.15.6:0/2369496083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 msgr2=0x563bb52414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb52414e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf60096980 tx=0x7faf60010c60 comp rx=0 tx=0).stop 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 -- 172.21.15.6:0/2369496083 shutdown_connections 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb50986b0 0x563bb5245e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb5242b90 0x563bb5236f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 --2- 172.21.15.6:0/2369496083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242720 0x563bb52414e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.222+0000 7faf8432bb80 1 -- 172.21.15.6:0/2369496083 >> 172.21.15.6:0/2369496083 conn(0x563bb5231f50 msgr2=0x563bb5240dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.223+0000 7faf8432bb80 1 -- 172.21.15.6:0/2369496083 shutdown_connections 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.223+0000 7faf8432bb80 1 -- 172.21.15.6:0/2369496083 wait complete. 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.223+0000 7faf8432bb80 1 -- start start 2023-11-17T00:10:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.223+0000 7faf8432bb80 1 Processor -- start 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 0x563bb524be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb5249630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb524a2f0 con 0x563bb5242720 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb524a460 con 0x563bb5242b90 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb52a8d90 con 0x563bb50986b0 2023-11-17T00:10:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf8432bb80 10 client.0 _ll_register_callbacks cb 0x7faf60099a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf69d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb5249630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 0x563bb524be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:51374/0 (socket says 172.21.15.6:51374) 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 -- 172.21.15.6:0/4082375793 learned_addr learned my addr 172.21.15.6:0/4082375793 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 0x563bb524be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53740/0 (socket says 172.21.15.6:53740) 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 -- 172.21.15.6:0/4082375793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 msgr2=0x563bb524be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 0x563bb524be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 -- 172.21.15.6:0/4082375793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 msgr2=0x563bb5249630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.224+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb5249630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf6ad9d700 1 -- 172.21.15.6:0/4082375793 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c00ec90 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf6ad9d700 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf640098e0 tx=0x7faf64005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4284cbb52f259c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6401d040 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faf64004910 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf6401e040 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf64005be0 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf8432bb80 10 client.25796 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.225+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563bb5154130 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6401e040 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf6400fbd0 con 0x563bb5242b90 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 10 client.25796 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 20 client.25796 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb52a9f60 0x563bb52ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 4 client.25796 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563bb515c600 con 0x563bb52a9f60 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.226+0000 7faf6a59c700 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb52a9f60 0x563bb52ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.227+0000 7faf6a59c700 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb52a9f60 0x563bb52ac370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0c988bd7b6291b server_cookie=fe09b84be82200af in_seq=0 out_seq=0 2023-11-17T00:10:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.227+0000 7faf51ffb700 10 client.25796 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf51ffb700 1 -- 172.21.15.6:0/4082375793 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563bb515c600 con 0x563bb52a9f60 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf51ffb700 10 client.25796 handle_command_reply: tid=0 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 client.25796 shutdown 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 msgr2=0x563bb5249b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf640098e0 tx=0x7faf64005750 comp rx=0 tx=0).stop 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 shutdown_connections 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563bb5242b90 0x563bb5249b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563bb50986b0 0x563bb524be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563bb52a9f60 0x563bb52ac370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 --2- 172.21.15.6:0/4082375793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563bb5242720 0x563bb5249630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf51ffb700 0 client.25796 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 >> 172.21.15.6:0/4082375793 conn(0x563bb5231f50 msgr2=0x563bb5241660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 shutdown_connections 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 1 -- 172.21.15.6:0/4082375793 wait complete. 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.248+0000 7faf8432bb80 20 client.25796 trim_cache size 0 max 16384 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:10:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 msgr2=0x7faf540a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 0x7faf540a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009c10 tx=0x7faf5c00b040 comp rx=0 tx=0).stop 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 msgr2=0x7faf6c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c198ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9f0 tx=0x7faf6000ed00 comp rx=0 tx=0).stop 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 shutdown_connections 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7faf540a1460 0x7faf540a3910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c108490 0x7faf6c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c102480 0x7faf6c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 --2- 172.21.15.6:0/1018917401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf6c102e30 0x7faf6c198ab0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 >> 172.21.15.6:0/1018917401 conn(0x7faf6c0fdf60 msgr2=0x7faf6c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 shutdown_connections 2023-11-17T00:10:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:06.249+0000 7faf709ff700 1 -- 172.21.15.6:0/1018917401 wait complete. 2023-11-17T00:10:06.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.427+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e8067af0 con 0x7f60e8101660 2023-11-17T00:10:06.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.428+0000 7f60cffff700 1 -- 172.21.15.6:0/2265140430 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f60dc09d0e0 con 0x7f60e8101660 2023-11-17T00:10:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:10:06.429 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:10:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.430+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 msgr2=0x7f60d00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.430+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 0x7f60d00a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60d8005e90 tx=0x7f60d8019040 comp rx=0 tx=0).stop 2023-11-17T00:10:06.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.430+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 msgr2=0x7f60e8194840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.430+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60dc012f90 tx=0x7f60dc00df80 comp rx=0 tx=0).stop 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 shutdown_connections 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60d00a13e0 0x7f60d00a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60e8105620 0x7f60e8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60e8100cc0 0x7f60e8194300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 --2- 172.21.15.6:0/2265140430 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60e8101660 0x7f60e8194840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 >> 172.21.15.6:0/2265140430 conn(0x7f60e80774d0 msgr2=0x7f60e80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 shutdown_connections 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:06.431+0000 7f60ed5ff700 1 -- 172.21.15.6:0/2265140430 wait complete. 2023-11-17T00:10:06.432 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:10:06.453 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:10:06.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:06 smithi006 ceph-mon[97840]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 25 op/s 2023-11-17T00:10:06.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:06 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2265140430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:06.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:06 smithi087 ceph-mon[131182]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 25 op/s 2023-11-17T00:10:06.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:06 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2265140430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:06.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:06 smithi186 ceph-mon[131487]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 KiB/s wr, 25 op/s 2023-11-17T00:10:06.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:06 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2265140430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:06.983 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:06.983 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:07.024 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:07.025 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:07.025 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 28 -lt 100 ']' 2023-11-17T00:10:07.025 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 28 + 1 2023-11-17T00:10:07.035 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=29 2023-11-17T00:10:07.036 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:07.052 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:07.052 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:07.054 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:07.055 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:07.055 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:07.938 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:07.938 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:08.064 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:08.064 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:08.083 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:08.083 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:08.083 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 29 -lt 100 ']' 2023-11-17T00:10:08.084 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 29 + 1 2023-11-17T00:10:08.120 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=30 2023-11-17T00:10:08.120 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:08.189 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:08.189 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:08.197 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:08.197 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:08.197 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:08.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:08 smithi087 ceph-mon[131182]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 17 op/s 2023-11-17T00:10:08.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:08 smithi186 ceph-mon[131487]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 17 op/s 2023-11-17T00:10:08.907 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:08 smithi006 ceph-mon[97840]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 17 op/s 2023-11-17T00:10:09.063 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:09.063 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:09.217 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:09.217 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:09.283 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:09.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:09.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 30 -lt 100 ']' 2023-11-17T00:10:09.284 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 30 + 1 2023-11-17T00:10:09.319 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=31 2023-11-17T00:10:09.383 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:09.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:09.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:09.385 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:09.385 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:09.385 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:10:09.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:10:09.828 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:09 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:10:09.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:09 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:10:09.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:10:09.897 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:09 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:10:10.080 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:10.143 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:10.314 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:10.638 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 31 -lt 100 ']' 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 31 + 1 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=32 2023-11-17T00:10:10.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:10.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2864192274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98105560 msgr2=0x7f0c9810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:10.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2864192274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98105560 0x7f0c9810d130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000b0d0 tx=0x7f0c9000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2864192274 shutdown_connections 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2864192274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98105560 0x7f0c9810d130 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2864192274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c98100c50 0x7f0c98101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2864192274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c981015f0 0x7f0c98104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2864192274 >> 172.21.15.6:0/2864192274 conn(0x7f0c98077500 msgr2=0x7f0c98077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2864192274 shutdown_connections 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.467+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2864192274 wait complete. 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.468+0000 7f0c9e1ff700 1 Processor -- start 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.468+0000 7f0c9e1ff700 1 -- start start 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c981015f0 0x7f0c98198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c98105560 0x7f0c9819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819c5d0 con 0x7f0c98100c50 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819c740 con 0x7f0c981015f0 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c9e1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819c900 con 0x7f0c98105560 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54272/0 (socket says 172.21.15.6:54272) 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c977fe700 1 -- 172.21.15.6:0/2174453028 learned_addr learned my addr 172.21.15.6:0/2174453028 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c981015f0 0x7f0c98198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.469+0000 7f0c97fff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c98105560 0x7f0c9819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 -- 172.21.15.6:0/2174453028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c98105560 msgr2=0x7f0c9819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c98105560 0x7f0c9819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 -- 172.21.15.6:0/2174453028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c981015f0 msgr2=0x7f0c98198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c981015f0 0x7f0c98198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 -- 172.21.15.6:0/2174453028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009d00 con 0x7f0c98100c50 2023-11-17T00:10:10.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9a0 tx=0x7f0c8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c94ff9700 1 -- 172.21.15.6:0/2174453028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88009790 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c94ff9700 1 -- 172.21.15.6:0/2174453028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c88010430 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98110360 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c94ff9700 1 -- 172.21.15.6:0/2174453028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88018680 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.470+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c981101d0 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.473+0000 7f0c94ff9700 1 -- 172.21.15.6:0/2174453028 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f0c880105a0 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.475+0000 7f0c94ff9700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 0x7f0c800a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.475+0000 7f0c94ff9700 1 -- 172.21.15.6:0/2174453028 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c88014070 con 0x7f0c98100c50 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.475+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 0x7f0c800a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.475+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 0x7f0c800a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c98199cf0 tx=0x7f0c8c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:10.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:10.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:10.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:10.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:10.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 Processor -- start 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 -- start start 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 0x5648b568f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b568fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5590060 con 0x5648b568ab60 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b55e28a0 con 0x5648b568f8b0 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.561+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b55dbd30 con 0x5648b568b470 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c96ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 0x5648b568f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b568fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54276/0 (socket says 172.21.15.6:54276) 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 -- 172.21.15.6:0/2262742648 learned_addr learned my addr 172.21.15.6:0/2262742648 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 -- 172.21.15.6:0/2262742648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 msgr2=0x5648b568f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 0x5648b568f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 -- 172.21.15.6:0/2262742648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 msgr2=0x5648b568fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b568fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 -- 172.21.15.6:0/2262742648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648b545d630 con 0x5648b568ab60 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 0x5648b568f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c97fff700 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b568fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.562+0000 7f0c977fe700 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0c88099f50 tx=0x7f0c880999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20191d5ea14a7d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.563+0000 7f0c7dffb700 1 -- 172.21.15.6:0/2262742648 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8809dff0 con 0x5648b568ab60 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.563+0000 7f0c7dffb700 1 -- 172.21.15.6:0/2262742648 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c880a2410 con 0x5648b568ab60 2023-11-17T00:10:10.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.563+0000 7f0c7dffb700 1 -- 172.21.15.6:0/2262742648 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880996b0 con 0x5648b568ab60 2023-11-17T00:10:10.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.563+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/2262742648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 msgr2=0x5648b568af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:10.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.563+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0c88099f50 tx=0x7f0c880999f0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/2262742648 shutdown_connections 2023-11-17T00:10:10.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568ab60 0x5648b568af30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b568fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/2262742648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568b470 0x5648b568f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/2262742648 >> 172.21.15.6:0/2262742648 conn(0x5648b5680360 msgr2=0x5648b5685b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:10.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/2262742648 shutdown_connections 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/2262742648 wait complete. 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 -- start start 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.564+0000 7f0cb1a34b80 1 Processor -- start 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5697020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b56942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b5694920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b557d8e0 con 0x5648b568f8b0 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b557d580 con 0x5648b568b470 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b545f920 con 0x5648b568ab60 2023-11-17T00:10:10.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0cb1a34b80 10 client.0 _ll_register_callbacks cb 0x5648b559b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b5694920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b5694920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54278/0 (socket says 172.21.15.6:54278) 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0c96ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b56942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.565+0000 7f0c97fff700 1 -- 172.21.15.6:0/3346097113 learned_addr learned my addr 172.21.15.6:0/3346097113 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c977fe700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5697020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 -- 172.21.15.6:0/3346097113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 msgr2=0x5648b5697020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5697020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 -- 172.21.15.6:0/3346097113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 msgr2=0x5648b5694920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b5694920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 -- 172.21.15.6:0/3346097113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c981101d0 con 0x5648b568b470 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c977fe700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5697020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b56942a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c005950 tx=0x7f0c8c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120eff3464edf6da server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c004030 con 0x5648b568b470 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648b567db20 con 0x5648b568b470 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.566+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c8c004190 con 0x5648b568b470 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8c004350 con 0x5648b568b470 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 10 client.25808 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:10.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5648b55a2190 con 0x5648b568b470 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648b567db20 con 0x5648b568b470 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c8c01d9c0 con 0x5648b568b470 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 20 client.25808 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56a48b0 0x5648b56a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 4 client.25808 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.567+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648b559e0a0 con 0x5648b56a48b0 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.568+0000 7f0c977fe700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56a48b0 0x5648b56a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.568+0000 7f0c977fe700 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56a48b0 0x5648b56a6cc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8b7533dd4d5237 server_cookie=f04a523dfd954889 in_seq=0 out_seq=0 2023-11-17T00:10:10.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.568+0000 7f0c76ffd700 10 client.25808 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0c76ffd700 1 -- 172.21.15.6:0/3346097113 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5648b559e0a0 con 0x5648b56a48b0 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0c76ffd700 10 client.25808 handle_command_reply: tid=0 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 client.25808 shutdown 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 msgr2=0x5648b56942a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b56942a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c005950 tx=0x7f0c8c0047c0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 shutdown_connections 2023-11-17T00:10:10.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56a48b0 0x5648b56a6cc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b5694920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5697020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/3346097113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b56942a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0c76ffd700 0 client.25808 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 >> 172.21.15.6:0/3346097113 conn(0x5648b5680360 msgr2=0x5648b5690d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.569+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 shutdown_connections 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.570+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/3346097113 wait complete. 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.570+0000 7f0cb1a34b80 20 client.25808 trim_cache size 0 max 16384 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.583+0000 7f0cb1a34b80 1 Processor -- start 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.583+0000 7f0cb1a34b80 1 -- start start 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5696bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b5685aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:10.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5685fe0 con 0x5648b568b470 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b56a6c90 con 0x5648b568f8b0 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b56948c0 con 0x5648b568ab60 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54288/0 (socket says 172.21.15.6:54288) 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 -- 172.21.15.6:0/875192603 learned_addr learned my addr 172.21.15.6:0/875192603 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c977fe700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5696bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c97fff700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b5685aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 -- 172.21.15.6:0/875192603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 msgr2=0x5648b5696bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5696bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 -- 172.21.15.6:0/875192603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 msgr2=0x5648b5685aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b5685aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c96ffd700 1 -- 172.21.15.6:0/875192603 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648b55a1ee0 con 0x5648b568b470 2023-11-17T00:10:10.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.584+0000 7f0c97fff700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b5685aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:10:10.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.585+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c016040 tx=0x7f0c8c01df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3cc2c7e13f63ad server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:10.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.585+0000 7f0c7e7fc700 1 -- 172.21.15.6:0/875192603 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c01d6c0 con 0x5648b568b470 2023-11-17T00:10:11.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.585+0000 7f0c7e7fc700 1 -- 172.21.15.6:0/875192603 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c8c004030 con 0x5648b568b470 2023-11-17T00:10:11.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.585+0000 7f0c7e7fc700 1 -- 172.21.15.6:0/875192603 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c003b10 con 0x5648b568b470 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/875192603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 msgr2=0x5648b5685560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c016040 tx=0x7f0c8c01df10 comp rx=0 tx=0).stop 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/875192603 shutdown_connections 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568f8b0 0x5648b5685aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5696bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/875192603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568b470 0x5648b5685560 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/875192603 >> 172.21.15.6:0/875192603 conn(0x5648b5680360 msgr2=0x5648b568d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:11.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/875192603 shutdown_connections 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/875192603 wait complete. 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 -- start start 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.586+0000 7f0cb1a34b80 1 Processor -- start 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b569a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b56f7cb0 con 0x5648b568f8b0 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b56f7df0 con 0x5648b568b470 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b56f7fb0 con 0x5648b568ab60 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0cb1a34b80 10 client.0 _ll_register_callbacks cb 0x7f0c8c01f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.587+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:11.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54300/0 (socket says 172.21.15.6:54300) 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42804/0 (socket says 172.21.15.6:42804) 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 -- 172.21.15.6:0/416719837 learned_addr learned my addr 172.21.15.6:0/416719837 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c96ffd700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b569a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 -- 172.21.15.6:0/416719837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 msgr2=0x5648b5699b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 -- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 msgr2=0x5648b569a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b569a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 -- 172.21.15.6:0/416719837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648b55a2190 con 0x5648b568f8b0 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c977fe700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c97fff700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000bf10 tx=0x7f0c9000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68ecf81197a6adc server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.588+0000 7f0c76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000e040 con 0x5648b568f8b0 2023-11-17T00:10:11.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c880a2410 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c90003a00 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c90003bc0 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 10 client.15750 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5648b53f2c60 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c880a2410 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c9000b100 con 0x5648b568f8b0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 20 client.15750 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56f96d0 0x5648b56fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 4 client.15750 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.589+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5648b55aa660 con 0x5648b56f96d0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.590+0000 7f0c977fe700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56f96d0 0x5648b56fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:11.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.590+0000 7f0c977fe700 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56f96d0 0x5648b56fbae0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6b93bc01b54d98 server_cookie=f21621a0cd9ee5ff in_seq=0 out_seq=0 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.590+0000 7f0c76ffd700 10 client.15750 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.590+0000 7f0c76ffd700 1 -- 172.21.15.6:0/416719837 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5648b55aa660 con 0x5648b56f96d0 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.590+0000 7f0c76ffd700 10 client.15750 handle_command_reply: tid=0 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 client.15750 shutdown 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 msgr2=0x5648b56f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000bf10 tx=0x7f0c9000bf40 comp rx=0 tx=0).stop 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 shutdown_connections 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5648b56f96d0 0x5648b56fbae0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648b568f8b0 0x5648b56f7770 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5648b568ab60 0x5648b5699b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0c76ffd700 0 client.15750 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:11.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 --2- 172.21.15.6:0/416719837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5648b568b470 0x5648b569a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 >> 172.21.15.6:0/416719837 conn(0x5648b5680360 msgr2=0x5648b5689960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:11.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 shutdown_connections 2023-11-17T00:10:11.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 1 -- 172.21.15.6:0/416719837 wait complete. 2023-11-17T00:10:11.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.591+0000 7f0cb1a34b80 20 client.15750 trim_cache size 0 max 16384 2023-11-17T00:10:11.391 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:10:11.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 msgr2=0x7f0c800a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:11.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 0x7f0c800a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c98199cf0 tx=0x7f0c8c009260 comp rx=0 tx=0).stop 2023-11-17T00:10:11.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 msgr2=0x7f0c981986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9a0 tx=0x7f0c8800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 shutdown_connections 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0c800a1460 0x7f0c800a3910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c98100c50 0x7f0c981986e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c98105560 0x7f0c9819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 --2- 172.21.15.6:0/2174453028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c981015f0 0x7f0c98198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 >> 172.21.15.6:0/2174453028 conn(0x7f0c98077500 msgr2=0x7f0c980feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 shutdown_connections 2023-11-17T00:10:11.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:10.592+0000 7f0c9e1ff700 1 -- 172.21.15.6:0/2174453028 wait complete. 2023-11-17T00:10:11.394 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:10 smithi006 ceph-mon[97840]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 25 op/s 2023-11-17T00:10:11.415 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:10 smithi186 ceph-mon[131487]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 25 op/s 2023-11-17T00:10:11.416 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:10 smithi087 ceph-mon[131182]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 25 op/s 2023-11-17T00:10:12.362 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:12.363 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:12.395 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:10:12.395 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:10:12.666 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:12.666 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:12.711 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:12.711 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:12.712 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 32 -lt 100 ']' 2023-11-17T00:10:12.712 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 32 + 1 2023-11-17T00:10:12.744 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=33 2023-11-17T00:10:12.744 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:12.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:12.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:12.772 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:12.772 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:12.773 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:12 smithi087 ceph-mon[131182]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:12 smithi186 ceph-mon[131487]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:12 smithi006 ceph-mon[97840]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:13.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:13.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:14.605 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:14.605 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:14.642 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:14.642 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:14.642 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 33 -lt 100 ']' 2023-11-17T00:10:14.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 33 + 1 2023-11-17T00:10:14.702 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=34 2023-11-17T00:10:14.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:14.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:14.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:15.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:14 smithi087 ceph-mon[131182]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:15.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:14 smithi186 ceph-mon[131487]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:15.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:14 smithi006 ceph-mon[97840]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-11-17T00:10:15.695 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:15.695 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:15.905 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:15.905 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:15.923 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:15.924 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:15.924 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 34 -lt 100 ']' 2023-11-17T00:10:15.924 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 34 + 1 2023-11-17T00:10:15.969 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=35 2023-11-17T00:10:15.969 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:15.998 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:15.998 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:16.001 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:16.002 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:16.002 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 -- 172.21.15.6:0/674928040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 msgr2=0x7fdf00105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 --2- 172.21.15.6:0/674928040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf00105670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0098f0 tx=0x7fdefc009c00 comp rx=0 tx=0).stop 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 -- 172.21.15.6:0/674928040 shutdown_connections 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 --2- 172.21.15.6:0/674928040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf0010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 --2- 172.21.15.6:0/674928040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf00105670 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 --2- 172.21.15.6:0/674928040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf00068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 -- 172.21.15.6:0/674928040 >> 172.21.15.6:0/674928040 conn(0x7fdf000763a0 msgr2=0x7fdf000767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 -- 172.21.15.6:0/674928040 shutdown_connections 2023-11-17T00:10:16.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.558+0000 7fdf075ff700 1 -- 172.21.15.6:0/674928040 wait complete. 2023-11-17T00:10:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.575+0000 7fdf075ff700 1 Processor -- start 2023-11-17T00:10:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.575+0000 7fdf075ff700 1 -- start start 2023-11-17T00:10:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf00100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf001a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001008b0 con 0x7fdf00067dc0 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a4af0 con 0x7fdf00068770 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf075ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a4c30 con 0x7fdf001052a0 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54312/0 (socket says 172.21.15.6:54312) 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf0539b700 1 -- 172.21.15.6:0/1168634328 learned_addr learned my addr 172.21.15.6:0/1168634328 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf00100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.576+0000 7fdf05b9c700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf001a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 -- 172.21.15.6:0/1168634328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 msgr2=0x7fdf001a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf001a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 -- 172.21.15.6:0/1168634328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 msgr2=0x7fdf00100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf00100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 -- 172.21.15.6:0/1168634328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009520 con 0x7fdf00067dc0 2023-11-17T00:10:16.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf05b9c700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf001a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:16.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf00100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:16.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf0539b700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009810 tx=0x7fdefc00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdef67fc700 1 -- 172.21.15.6:0/1168634328 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc01c070 con 0x7fdf00067dc0 2023-11-17T00:10:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdef67fc700 1 -- 172.21.15.6:0/1168634328 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc005040 con 0x7fdf00067dc0 2023-11-17T00:10:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdef67fc700 1 -- 172.21.15.6:0/1168634328 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc004a10 con 0x7fdf00067dc0 2023-11-17T00:10:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf00110370 con 0x7fdf00067dc0 2023-11-17T00:10:16.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.577+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001101e0 con 0x7fdf00067dc0 2023-11-17T00:10:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.581+0000 7fdef67fc700 1 -- 172.21.15.6:0/1168634328 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fdefc020070 con 0x7fdf00067dc0 2023-11-17T00:10:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.582+0000 7fdef67fc700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 0x7fdeec0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.582+0000 7fdef67fc700 1 -- 172.21.15.6:0/1168634328 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdefc0cdfe0 con 0x7fdf00067dc0 2023-11-17T00:10:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.583+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 0x7fdeec0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.583+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 0x7fdeec0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf00101330 tx=0x7fdef0009310 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.637 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:16.637 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:16.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.718+0000 7fdf1ae74b80 1 Processor -- start 2023-11-17T00:10:16.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 -- start start 2023-11-17T00:10:16.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f32d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f32dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f22e0e0 con 0x55832f3294f0 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f280920 con 0x55832f32d930 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f279db0 con 0x55832f328be0 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf04b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f32d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42836/0 (socket says 172.21.15.6:42836) 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.719+0000 7fdf0539b700 1 -- 172.21.15.6:0/446840245 learned_addr learned my addr 172.21.15.6:0/446840245 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf05b9c700 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f32dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 -- 172.21.15.6:0/446840245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 msgr2=0x55832f32dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f32dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 -- 172.21.15.6:0/446840245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 msgr2=0x55832f32d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f32d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 -- 172.21.15.6:0/446840245 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832f0fb6b0 con 0x55832f328be0 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.720+0000 7fdf0539b700 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdefc097040 tx=0x7fdefc09a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00fedeeeb5e83d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.721+0000 7fdeeb7fe700 1 -- 172.21.15.6:0/446840245 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc09eb70 con 0x55832f328be0 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.721+0000 7fdeeb7fe700 1 -- 172.21.15.6:0/446840245 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc0a2f00 con 0x55832f328be0 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.721+0000 7fdeeb7fe700 1 -- 172.21.15.6:0/446840245 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc09a290 con 0x55832f328be0 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.721+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/446840245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 msgr2=0x55832f328fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.721+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdefc097040 tx=0x7fdefc09a570 comp rx=0 tx=0).stop 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/446840245 shutdown_connections 2023-11-17T00:10:16.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f32dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f328be0 0x55832f328fb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/446840245 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f32d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/446840245 >> 172.21.15.6:0/446840245 conn(0x55832f31e3e0 msgr2=0x55832f323bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/446840245 shutdown_connections 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/446840245 wait complete. 2023-11-17T00:10:16.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.722+0000 7fdf1ae74b80 1 -- start start 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 Processor -- start 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f334fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f3327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f21b960 con 0x55832f328be0 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f21b600 con 0x55832f3294f0 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f0fd9a0 con 0x55832f32d930 2023-11-17T00:10:16.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.723+0000 7fdf1ae74b80 10 client.0 _ll_register_callbacks cb 0x55832f239800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f334fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f334fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54344/0 (socket says 172.21.15.6:54344) 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40752/0 (socket says 172.21.15.6:40752) 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf0539b700 1 -- 172.21.15.6:0/1009453460 learned_addr learned my addr 172.21.15.6:0/1009453460 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf05b9c700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f3327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 -- 172.21.15.6:0/1009453460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 msgr2=0x55832f3327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f3327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 -- 172.21.15.6:0/1009453460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 msgr2=0x55832f334fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f334fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 -- 172.21.15.6:0/1009453460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001101e0 con 0x55832f3294f0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.724+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdef0003aa0 tx=0x7fdef00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6803898de3756b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef001fa90 con 0x55832f3294f0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdef001fbf0 con 0x55832f3294f0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55832f31bba0 con 0x55832f3294f0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef001fdb0 con 0x55832f3294f0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdf1ae74b80 10 client.25823 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.725+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55832f240210 con 0x55832f3294f0 2023-11-17T00:10:16.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55832f31bba0 con 0x55832f3294f0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdef0027470 con 0x55832f3294f0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 20 client.25823 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f3426c0 0x55832f344ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 4 client.25823 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55832f23c120 con 0x55832f3426c0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.726+0000 7fdf0539b700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f3426c0 0x55832f344ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.727+0000 7fdee8ff9700 10 client.25823 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.727+0000 7fdf0539b700 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f3426c0 0x55832f344ad0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f844efa70c297e server_cookie=484fdfeac6ad0a0e in_seq=0 out_seq=0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.727+0000 7fdee8ff9700 1 -- 172.21.15.6:0/1009453460 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55832f23c120 con 0x55832f3426c0 2023-11-17T00:10:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdee8ff9700 10 client.25823 handle_command_reply: tid=0 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 client.25823 shutdown 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 msgr2=0x55832f332290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdef0003aa0 tx=0x7fdef00047c0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 shutdown_connections 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f3426c0 0x55832f344ad0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f334fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f3327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdee8ff9700 0 client.25823 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/1009453460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f3294f0 0x55832f332290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 >> 172.21.15.6:0/1009453460 conn(0x55832f31e3e0 msgr2=0x55832f322e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 shutdown_connections 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/1009453460 wait complete. 2023-11-17T00:10:16.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.728+0000 7fdf1ae74b80 20 client.25823 trim_cache size 0 max 16384 2023-11-17T00:10:16.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.746+0000 7fdf1ae74b80 1 Processor -- start 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 -- start start 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f3294f0 0x55832f334b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f33e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f33fcf0 con 0x55832f328be0 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f3331c0 con 0x55832f32d930 2023-11-17T00:10:16.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f3307f0 con 0x55832f3294f0 2023-11-17T00:10:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54356/0 (socket says 172.21.15.6:54356) 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf0539b700 1 -- 172.21.15.6:0/3473286635 learned_addr learned my addr 172.21.15.6:0/3473286635 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf05b9c700 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f33e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.747+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f3294f0 0x55832f334b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 -- 172.21.15.6:0/3473286635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f3294f0 msgr2=0x55832f334b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f3294f0 0x55832f334b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 -- 172.21.15.6:0/3473286635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 msgr2=0x55832f33e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f33e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 -- 172.21.15.6:0/3473286635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832f23ff60 con 0x55832f328be0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdf0539b700 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdefc09aad0 tx=0x7fdefc019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286ad053577b5dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdeebfff700 1 -- 172.21.15.6:0/3473286635 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc098f60 con 0x55832f328be0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdeebfff700 1 -- 172.21.15.6:0/3473286635 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc0b3240 con 0x55832f328be0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.748+0000 7fdeebfff700 1 -- 172.21.15.6:0/3473286635 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc09ada0 con 0x55832f328be0 2023-11-17T00:10:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/3473286635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 msgr2=0x55832f339840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdefc09aad0 tx=0x7fdefc019040 comp rx=0 tx=0).stop 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/3473286635 shutdown_connections 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f328be0 0x55832f339840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f32d930 0x55832f33e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/3473286635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f3294f0 0x55832f334b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/3473286635 >> 172.21.15.6:0/3473286635 conn(0x55832f31e3e0 msgr2=0x55832f33dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/3473286635 shutdown_connections 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.749+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/3473286635 wait complete. 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 -- start start 2023-11-17T00:10:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 Processor -- start 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 0x55832f337eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f3383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f338930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.750+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f396590 con 0x55832f3294f0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f3966d0 con 0x55832f328be0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf1ae74b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832f396810 con 0x55832f32d930 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf1ae74b80 10 client.0 _ll_register_callbacks cb 0x7fdef0003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf04b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f3383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf05b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f338930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf04b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f3383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54362/0 (socket says 172.21.15.6:54362) 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf04b9a700 1 -- 172.21.15.6:0/2744593159 learned_addr learned my addr 172.21.15.6:0/2744593159 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 0x55832f337eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 -- 172.21.15.6:0/2744593159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 msgr2=0x55832f338930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f338930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 -- 172.21.15.6:0/2744593159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 msgr2=0x55832f3383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f3383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.751+0000 7fdf0539b700 1 -- 172.21.15.6:0/2744593159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc0b2040 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdf0539b700 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 0x55832f337eb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdefc09a840 tx=0x7fdefc09acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3abd1423bc2febf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc098f60 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdefc0a2f00 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdefc09ada0 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc09afc0 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdf1ae74b80 10 client.25829 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55832f090ce0 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc0be020 con 0x55832f328be0 2023-11-17T00:10:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.752+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdefc0a2a70 con 0x55832f328be0 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 10 client.25829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 20 client.25829 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f397a20 0x55832f399e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 4 client.25829 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55832f2486e0 con 0x55832f397a20 2023-11-17T00:10:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f397a20 0x55832f399e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdf04b9a700 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f397a20 0x55832f399e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7a07b5173b9f0c server_cookie=eedae13b0c8a1332 in_seq=0 out_seq=0 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.753+0000 7fdee8ff9700 10 client.25829 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.754+0000 7fdee8ff9700 1 -- 172.21.15.6:0/2744593159 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55832f2486e0 con 0x55832f397a20 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.754+0000 7fdee8ff9700 10 client.25829 handle_command_reply: tid=0 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.754+0000 7fdf1ae74b80 1 client.25829 shutdown 2023-11-17T00:10:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 msgr2=0x55832f337eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 0x55832f337eb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdefc09a840 tx=0x7fdefc09acc0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 shutdown_connections 2023-11-17T00:10:16.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55832f32d930 0x55832f338930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55832f328be0 0x55832f337eb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55832f397a20 0x55832f399e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 --2- 172.21.15.6:0/2744593159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55832f3294f0 0x55832f3383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 >> 172.21.15.6:0/2744593159 conn(0x55832f31e3e0 msgr2=0x55832f33e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdee8ff9700 0 client.25829 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 shutdown_connections 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 1 -- 172.21.15.6:0/2744593159 wait complete. 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.755+0000 7fdf1ae74b80 20 client.25829 trim_cache size 0 max 16384 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "010ebc49-f8d2-4093-9627-a566998bb22f", 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:10:16.764 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 msgr2=0x7fdeec0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 0x7fdeec0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf00101330 tx=0x7fdef0009310 comp rx=0 tx=0).stop 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 msgr2=0x7fdf000ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009810 tx=0x7fdefc00b560 comp rx=0 tx=0).stop 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 shutdown_connections 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdeec0a1460 0x7fdeec0a3910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf00067dc0 0x7fdf000ffb40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf001052a0 0x7fdf001a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 --2- 172.21.15.6:0/1168634328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf00068770 0x7fdf00100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:16.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 >> 172.21.15.6:0/1168634328 conn(0x7fdf000763a0 msgr2=0x7fdf000fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:16.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 shutdown_connections 2023-11-17T00:10:16.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:16.756+0000 7fdf075ff700 1 -- 172.21.15.6:0/1168634328 wait complete. 2023-11-17T00:10:16.824 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:10:17.065 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:17.066 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:17.091 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:17.091 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:17.092 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 35 -lt 100 ']' 2023-11-17T00:10:17.092 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 35 + 1 2023-11-17T00:10:17.100 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:16 smithi087 ceph-mon[131182]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 25 op/s 2023-11-17T00:10:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:16 smithi186 ceph-mon[131487]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 25 op/s 2023-11-17T00:10:17.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=36 2023-11-17T00:10:17.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:17.168 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:17.171 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:17.171 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:17.171 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:17.172 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:17.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:16 smithi006 ceph-mon[97840]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 25 op/s 2023-11-17T00:10:17.636 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:17.637 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:17.741 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:17.741 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:17.765 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:17.765 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:17.765 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 36 -lt 100 ']' 2023-11-17T00:10:17.766 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 36 + 1 2023-11-17T00:10:17.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=37 2023-11-17T00:10:17.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:17.844 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:17.845 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:17.847 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:17.848 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:17.848 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:18.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:17 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:10:18.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:17 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:17 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:10:18.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:17 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:17 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:10:18.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:17 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:17 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:10:18.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:17 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:17 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:10:18.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:17 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.187 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:17 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:10:18.187 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:17 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:18.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:18.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:18.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:18.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:18.832 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:18.833 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:18.833 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 37 -lt 100 ']' 2023-11-17T00:10:18.833 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 37 + 1 2023-11-17T00:10:18.917 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=38 2023-11-17T00:10:18.918 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:18.954 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:18.954 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:18.957 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:18.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:18.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:18 smithi087 ceph-mon[131182]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-17T00:10:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:18 smithi186 ceph-mon[131487]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-17T00:10:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:18 smithi006 ceph-mon[97840]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-17T00:10:19.678 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:19.678 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:19.930 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:19.930 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:19.957 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:19.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:19.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 38 -lt 100 ']' 2023-11-17T00:10:19.958 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 38 + 1 2023-11-17T00:10:20.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=39 2023-11-17T00:10:20.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:20.261 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:20.262 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:20.265 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:20.266 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:20.266 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:20.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.340+0000 7fae6a5ff700 1 -- 172.21.15.6:0/2540429412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 msgr2=0x7fae64101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.340+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/2540429412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64101020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae540099c0 tx=0x7fae54009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 -- 172.21.15.6:0/2540429412 shutdown_connections 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/2540429412 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64101020 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/2540429412 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 0x7fae6410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/2540429412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 0x7fae64104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 -- 172.21.15.6:0/2540429412 >> 172.21.15.6:0/2540429412 conn(0x7fae64077500 msgr2=0x7fae64077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 -- 172.21.15.6:0/2540429412 shutdown_connections 2023-11-17T00:10:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.341+0000 7fae6a5ff700 1 -- 172.21.15.6:0/2540429412 wait complete. 2023-11-17T00:10:20.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 Processor -- start 2023-11-17T00:10:20.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 -- start start 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 0x7fae64073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 0x7fae641a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64073b60 con 0x7fae64100c50 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a4ac0 con 0x7fae64105560 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.342+0000 7fae6a5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a4c00 con 0x7fae641015f0 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52472/0 (socket says 172.21.15.6:52472) 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 -- 172.21.15.6:0/120989815 learned_addr learned my addr 172.21.15.6:0/120989815 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae637fe700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 0x7fae64073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae68b9c700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 0x7fae641a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 -- 172.21.15.6:0/120989815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 msgr2=0x7fae64073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 0x7fae64073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 -- 172.21.15.6:0/120989815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 msgr2=0x7fae641a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 0x7fae641a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 -- 172.21.15.6:0/120989815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540095f0 con 0x7fae64100c50 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.343+0000 7fae63fff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae5400b580 tx=0x7fae5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.344+0000 7fae617fa700 1 -- 172.21.15.6:0/120989815 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401d070 con 0x7fae64100c50 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.344+0000 7fae617fa700 1 -- 172.21.15.6:0/120989815 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae54005050 con 0x7fae64100c50 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.344+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae64110360 con 0x7fae64100c50 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.344+0000 7fae617fa700 1 -- 172.21.15.6:0/120989815 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54004a40 con 0x7fae64100c50 2023-11-17T00:10:20.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.344+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641101d0 con 0x7fae64100c50 2023-11-17T00:10:20.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.347+0000 7fae617fa700 1 -- 172.21.15.6:0/120989815 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fae54005430 con 0x7fae64100c50 2023-11-17T00:10:20.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.348+0000 7fae617fa700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 0x7fae4c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.348+0000 7fae617fa700 1 -- 172.21.15.6:0/120989815 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fae540cdfd0 con 0x7fae64100c50 2023-11-17T00:10:20.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.349+0000 7fae637fe700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 0x7fae4c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.349+0000 7fae637fe700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 0x7fae4c0a3910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae640745e0 tx=0x7fae58009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.451+0000 7fae7de27b80 1 Processor -- start 2023-11-17T00:10:20.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.451+0000 7fae7de27b80 1 -- start start 2023-11-17T00:10:20.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a685470 0x55cd0a689370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a6898b0 0x55cd0a689d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a58a060 con 0x55cd0a685470 2023-11-17T00:10:20.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a5dc8a0 con 0x55cd0a684b60 2023-11-17T00:10:20.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae7de27b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a5d5d30 con 0x55cd0a6898b0 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57572/0 (socket says 172.21.15.6:57572) 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 -- 172.21.15.6:0/2963648418 learned_addr learned my addr 172.21.15.6:0/2963648418 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae637fe700 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a685470 0x55cd0a689370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae68b9c700 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a6898b0 0x55cd0a689d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 -- 172.21.15.6:0/2963648418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a6898b0 msgr2=0x55cd0a689d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a6898b0 0x55cd0a689d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 -- 172.21.15.6:0/2963648418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a685470 msgr2=0x55cd0a689370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.452+0000 7fae63fff700 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a685470 0x55cd0a689370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.453+0000 7fae63fff700 1 -- 172.21.15.6:0/2963648418 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0a457630 con 0x55cd0a684b60 2023-11-17T00:10:20.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.453+0000 7fae63fff700 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c00 tx=0x7fae5409ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3467128e644b01be server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.453+0000 7fae4a7fc700 1 -- 172.21.15.6:0/2963648418 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540ba050 con 0x55cd0a684b60 2023-11-17T00:10:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.453+0000 7fae4a7fc700 1 -- 172.21.15.6:0/2963648418 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae540a2f50 con 0x55cd0a684b60 2023-11-17T00:10:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.453+0000 7fae4a7fc700 1 -- 172.21.15.6:0/2963648418 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5409a2e0 con 0x55cd0a684b60 2023-11-17T00:10:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 -- 172.21.15.6:0/2963648418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 msgr2=0x55cd0a684f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c00 tx=0x7fae5409ac10 comp rx=0 tx=0).stop 2023-11-17T00:10:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 -- 172.21.15.6:0/2963648418 shutdown_connections 2023-11-17T00:10:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a6898b0 0x55cd0a689d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a684b60 0x55cd0a684f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 --2- 172.21.15.6:0/2963648418 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a685470 0x55cd0a689370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 -- 172.21.15.6:0/2963648418 >> 172.21.15.6:0/2963648418 conn(0x55cd0a67a360 msgr2=0x55cd0a67fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 -- 172.21.15.6:0/2963648418 shutdown_connections 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.454+0000 7fae7de27b80 1 -- 172.21.15.6:0/2963648418 wait complete. 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 -- start start 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 Processor -- start 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a68e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a68e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.455+0000 7fae7de27b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a5778e0 con 0x55cd0a684b60 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae7de27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a577580 con 0x55cd0a6898b0 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae7de27b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a459920 con 0x55cd0a685470 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae7de27b80 10 client.0 _ll_register_callbacks cb 0x55cd0a595780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52488/0 (socket says 172.21.15.6:52488) 2023-11-17T00:10:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 -- 172.21.15.6:0/3149184388 learned_addr learned my addr 172.21.15.6:0/3149184388 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae637fe700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a68e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae68b9c700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a68e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 -- 172.21.15.6:0/3149184388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 msgr2=0x55cd0a68e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a68e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 -- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 msgr2=0x55cd0a68e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a68e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 -- 172.21.15.6:0/3149184388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641101d0 con 0x55cd0a684b60 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae637fe700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a68e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae63fff700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae54097600 tx=0x7fae54096fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632b0d8eb26384a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.456+0000 7fae437fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54098fb0 con 0x55cd0a684b60 2023-11-17T00:10:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fae540b3270 con 0x55cd0a684b60 2023-11-17T00:10:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd0a677b20 con 0x55cd0a684b60 2023-11-17T00:10:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5409b000 con 0x55cd0a684b60 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 10 client.15780 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cd0a59c190 con 0x55cd0a684b60 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd0a677b20 con 0x55cd0a684b60 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae5400fad0 con 0x55cd0a684b60 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 20 client.15780 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a69e040 0x55cd0a6a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 4 client.15780 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.457+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd0a5980a0 con 0x55cd0a69e040 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.458+0000 7fae637fe700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a69e040 0x55cd0a6a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.458+0000 7fae437fe700 10 client.15780 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.458+0000 7fae637fe700 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a69e040 0x55cd0a6a0420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03ad2c54adf63ac server_cookie=f076ae103d64496a in_seq=0 out_seq=0 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae437fe700 1 -- 172.21.15.6:0/3149184388 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd0a5980a0 con 0x55cd0a69e040 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae437fe700 10 client.15780 handle_command_reply: tid=0 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 client.15780 shutdown 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 msgr2=0x55cd0a690f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae54097600 tx=0x7fae54096fa0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 shutdown_connections 2023-11-17T00:10:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a69e040 0x55cd0a6a0420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a690f80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a68e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae437fe700 0 client.15780 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3149184388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a68e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 >> 172.21.15.6:0/3149184388 conn(0x55cd0a67a360 msgr2=0x55cd0a67ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 shutdown_connections 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 1 -- 172.21.15.6:0/3149184388 wait complete. 2023-11-17T00:10:20.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.460+0000 7fae7de27b80 20 client.15780 trim_cache size 0 max 16384 2023-11-17T00:10:20.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.479+0000 7fae7de27b80 1 Processor -- start 2023-11-17T00:10:20.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.479+0000 7fae7de27b80 1 -- start start 2023-11-17T00:10:20.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a685470 0x55cd0a690ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 0x55cd0a67f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a6883d0 con 0x55cd0a6898b0 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a679ee0 con 0x55cd0a685470 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae7de27b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a68ecf0 con 0x55cd0a684b60 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae68b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 0x55cd0a67f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57198/0 (socket says 172.21.15.6:57198) 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae68b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 0x55cd0a67f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52492/0 (socket says 172.21.15.6:52492) 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 -- 172.21.15.6:0/1363862329 learned_addr learned my addr 172.21.15.6:0/1363862329 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae637fe700 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a685470 0x55cd0a690ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 -- 172.21.15.6:0/1363862329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a685470 msgr2=0x55cd0a690ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a685470 0x55cd0a690ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 -- 172.21.15.6:0/1363862329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 msgr2=0x55cd0a67f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.480+0000 7fae63fff700 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 0x55cd0a67f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.481+0000 7fae63fff700 1 -- 172.21.15.6:0/1363862329 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0a67bb00 con 0x55cd0a684b60 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.481+0000 7fae63fff700 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae5409a890 tx=0x7fae5409a600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c37f095e848fe04 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.481+0000 7fae4affd700 1 -- 172.21.15.6:0/1363862329 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540ba040 con 0x55cd0a684b60 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.481+0000 7fae4affd700 1 -- 172.21.15.6:0/1363862329 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5400fe90 con 0x55cd0a684b60 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.481+0000 7fae4affd700 1 -- 172.21.15.6:0/1363862329 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540856b0 con 0x55cd0a684b60 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 -- 172.21.15.6:0/1363862329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 msgr2=0x55cd0a695750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae5409a890 tx=0x7fae5409a600 comp rx=0 tx=0).stop 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 -- 172.21.15.6:0/1363862329 shutdown_connections 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a6898b0 0x55cd0a67f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a684b60 0x55cd0a695750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 --2- 172.21.15.6:0/1363862329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a685470 0x55cd0a690ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 -- 172.21.15.6:0/1363862329 >> 172.21.15.6:0/1363862329 conn(0x55cd0a67a360 msgr2=0x55cd0a688890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 -- 172.21.15.6:0/1363862329 shutdown_connections 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.482+0000 7fae7de27b80 1 -- 172.21.15.6:0/1363862329 wait complete. 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.483+0000 7fae7de27b80 1 -- start start 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.483+0000 7fae7de27b80 1 Processor -- start 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a691c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a6921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a6f0e80 con 0x55cd0a684b60 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a6f0fc0 con 0x55cd0a6898b0 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0a6f1180 con 0x55cd0a685470 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae7de27b80 10 client.0 _ll_register_callbacks cb 0x7fae580096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52504/0 (socket says 172.21.15.6:52504) 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 -- 172.21.15.6:0/3770308705 learned_addr learned my addr 172.21.15.6:0/3770308705 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae68b9c700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a6921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae637fe700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a691c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 -- 172.21.15.6:0/3770308705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 msgr2=0x55cd0a691c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a691c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 -- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 msgr2=0x55cd0a6921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a6921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.484+0000 7fae63fff700 1 -- 172.21.15.6:0/3770308705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd0a59c190 con 0x55cd0a684b60 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae63fff700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fae540cb930 tx=0x7fae5409a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcd26c5ac27fd4a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540ba050 con 0x55cd0a684b60 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fae5409b000 con 0x55cd0a684b60 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae540a2f50 con 0x55cd0a684b60 2023-11-17T00:10:20.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5400fad0 con 0x55cd0a684b60 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 10 client.15786 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cd0a403de0 con 0x55cd0a684b60 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae54083030 con 0x55cd0a684b60 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae540a2bc0 con 0x55cd0a684b60 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.485+0000 7fae7de27b80 20 client.15786 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a6f23e0 0x55cd0a6f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae7de27b80 4 client.15786 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd0a5a4660 con 0x55cd0a6f23e0 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae637fe700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a6f23e0 0x55cd0a6f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae437fe700 10 client.15786 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.486+0000 7fae637fe700 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a6f23e0 0x55cd0a6f47c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cc23f7cb6c64ee server_cookie=8a0932ea9bba171 in_seq=0 out_seq=0 2023-11-17T00:10:20.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae437fe700 1 -- 172.21.15.6:0/3770308705 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cd0a5a4660 con 0x55cd0a6f23e0 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae437fe700 10 client.15786 handle_command_reply: tid=0 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 client.15786 shutdown 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 msgr2=0x55cd0a691730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fae540cb930 tx=0x7fae5409a740 comp rx=0 tx=0).stop 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 shutdown_connections 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55cd0a6f23e0 0x55cd0a6f47c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cd0a684b60 0x55cd0a691730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55cd0a6898b0 0x55cd0a6921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 --2- 172.21.15.6:0/3770308705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cd0a685470 0x55cd0a691c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae437fe700 0 client.15786 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 >> 172.21.15.6:0/3770308705 conn(0x55cd0a67a360 msgr2=0x55cd0a683960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 shutdown_connections 2023-11-17T00:10:20.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 1 -- 172.21.15.6:0/3770308705 wait complete. 2023-11-17T00:10:20.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.487+0000 7fae7de27b80 20 client.15786 trim_cache size 0 max 16384 2023-11-17T00:10:20.581 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:20.581 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:10:20.581 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:10:20.581 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:20.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 msgr2=0x7fae4c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 0x7fae4c0a3910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae640745e0 tx=0x7fae58009260 comp rx=0 tx=0).stop 2023-11-17T00:10:20.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 msgr2=0x7fae64072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae5400b580 tx=0x7fae5400f690 comp rx=0 tx=0).stop 2023-11-17T00:10:20.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 shutdown_connections 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fae4c0a1460 0x7fae4c0a3910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fae64100c50 0x7fae64072df0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae64105560 0x7fae641a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 --2- 172.21.15.6:0/120989815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae641015f0 0x7fae64073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 >> 172.21.15.6:0/120989815 conn(0x7fae64077500 msgr2=0x7fae640fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 shutdown_connections 2023-11-17T00:10:20.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:20.488+0000 7fae6a5ff700 1 -- 172.21.15.6:0/120989815 wait complete. 2023-11-17T00:10:20.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:10:20.583 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:10:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 -- 172.21.15.6:0/54614374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 msgr2=0x7fc81c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 --2- 172.21.15.6:0/54614374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1006b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0099c0 tx=0x7fc80c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 -- 172.21.15.6:0/54614374 shutdown_connections 2023-11-17T00:10:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 --2- 172.21.15.6:0/54614374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1006b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 --2- 172.21.15.6:0/54614374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c105320 0x7fc81c10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 --2- 172.21.15.6:0/54614374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c100c80 0x7fc81c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 -- 172.21.15.6:0/54614374 >> 172.21.15.6:0/54614374 conn(0x7fc81c0fbb70 msgr2=0x7fc81c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 -- 172.21.15.6:0/54614374 shutdown_connections 2023-11-17T00:10:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.669+0000 7fc820d9d700 1 -- 172.21.15.6:0/54614374 wait complete. 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.670+0000 7fc820d9d700 1 Processor -- start 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 -- start start 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c100c80 0x7fc81c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c105320 0x7fc81c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c19d3c0 con 0x7fc81c1002e0 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c19d530 con 0x7fc81c105320 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc820d9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c19d6f0 con 0x7fc81c100c80 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52508/0 (socket says 172.21.15.6:52508) 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc81b7fe700 1 -- 172.21.15.6:0/325428396 learned_addr learned my addr 172.21.15.6:0/325428396 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc81bfff700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c105320 0x7fc81c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.671+0000 7fc81affd700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c100c80 0x7fc81c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 -- 172.21.15.6:0/325428396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c100c80 msgr2=0x7fc81c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c100c80 0x7fc81c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 -- 172.21.15.6:0/325428396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c105320 msgr2=0x7fc81c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c105320 0x7fc81c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 -- 172.21.15.6:0/325428396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810009520 con 0x7fc81c1002e0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b580 tx=0x7fc80c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc818ff9700 1 -- 172.21.15.6:0/325428396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c01d070 con 0x7fc81c1002e0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c0095f0 con 0x7fc81c1002e0 2023-11-17T00:10:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc818ff9700 1 -- 172.21.15.6:0/325428396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c005050 con 0x7fc81c1002e0 2023-11-17T00:10:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c110130 con 0x7fc81c1002e0 2023-11-17T00:10:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.672+0000 7fc818ff9700 1 -- 172.21.15.6:0/325428396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c004a60 con 0x7fc81c1002e0 2023-11-17T00:10:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.676+0000 7fc818ff9700 1 -- 172.21.15.6:0/325428396 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fc80c005430 con 0x7fc81c1002e0 2023-11-17T00:10:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.677+0000 7fc8359ffb80 1 Processor -- start 2023-11-17T00:10:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.677+0000 7fc8359ffb80 1 -- start start 2023-11-17T00:10:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 0x55bc954f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954edcf0 0x55bc954ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc953f3000 con 0x55bc954edcf0 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc95445840 con 0x55bc954f3990 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9543ecd0 con 0x55bc953496b0 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc818ff9700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 0x7fc8040a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc818ff9700 1 -- 172.21.15.6:0/325428396 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc80c0cdff0 con 0x7fc81c1002e0 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 0x55bc954f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954edcf0 0x55bc954ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 0x55bc954f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57246/0 (socket says 172.21.15.6:57246) 2023-11-17T00:10:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81bfff700 1 -- 172.21.15.6:0/338150632 learned_addr learned my addr 172.21.15.6:0/338150632 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.678+0000 7fc81bfff700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 0x7fc8040a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 -- 172.21.15.6:0/338150632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 msgr2=0x55bc954f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 0x55bc954f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 -- 172.21.15.6:0/338150632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954edcf0 msgr2=0x55bc954ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954edcf0 0x55bc954ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 -- 172.21.15.6:0/338150632 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc952c05d0 con 0x55bc954f3990 2023-11-17T00:10:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954ed7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0cb270 tx=0x7fc80c09a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdcc35a45c7d7eed server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc81bfff700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 0x7fc8040a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814009100 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc801ffb700 1 -- 172.21.15.6:0/338150632 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c09ebe0 con 0x55bc954f3990 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc801ffb700 1 -- 172.21.15.6:0/338150632 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c0a2f70 con 0x55bc954f3990 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.679+0000 7fc801ffb700 1 -- 172.21.15.6:0/338150632 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0a2d20 con 0x55bc954f3990 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 -- 172.21.15.6:0/338150632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 msgr2=0x55bc954ed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954ed7b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0cb270 tx=0x7fc80c09a770 comp rx=0 tx=0).stop 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 -- 172.21.15.6:0/338150632 shutdown_connections 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954edcf0 0x55bc954ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc953496b0 0x55bc954f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/338150632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954ed7b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 -- 172.21.15.6:0/338150632 >> 172.21.15.6:0/338150632 conn(0x55bc954e31a0 msgr2=0x55bc954e83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 -- 172.21.15.6:0/338150632 shutdown_connections 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.680+0000 7fc8359ffb80 1 -- 172.21.15.6:0/338150632 wait complete. 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.681+0000 7fc8359ffb80 1 -- start start 2023-11-17T00:10:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.681+0000 7fc8359ffb80 1 Processor -- start 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.681+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.681+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc953e0880 con 0x55bc954f3990 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc953e0520 con 0x55bc953496b0 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc952c28c0 con 0x55bc954edcf0 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc8359ffb80 10 client.0 _ll_register_callbacks cb 0x55bc953fe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57642/0 (socket says 172.21.15.6:57642) 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52514/0 (socket says 172.21.15.6:52514) 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 -- 172.21.15.6:0/3574142087 learned_addr learned my addr 172.21.15.6:0/3574142087 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 -- 172.21.15.6:0/3574142087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 msgr2=0x55bc954fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 -- 172.21.15.6:0/3574142087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 msgr2=0x55bc954f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81b7fe700 1 -- 172.21.15.6:0/3574142087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c110130 con 0x55bc953496b0 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.682+0000 7fc81bfff700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc80c084300 tx=0x7fc80c09a460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd3e5bc99d2183 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c098f50 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc80c09ee40 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc954e0960 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80c0a2ae0 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc8359ffb80 10 client.25859 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bc95405130 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c0be080 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc80c0cb550 con 0x55bc953496b0 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.683+0000 7fc8359ffb80 10 client.25859 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc8359ffb80 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc8359ffb80 20 client.25859 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc955069a0 0x55bc95508db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc8359ffb80 4 client.25859 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc95401040 con 0x55bc955069a0 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc81affd700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc955069a0 0x55bc95508db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc81affd700 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc955069a0 0x55bc95508db0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37225fbb06ed18e8 server_cookie=d758fedeca2936d8 in_seq=0 out_seq=0 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.684+0000 7fc7faffd700 10 client.25859 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc7faffd700 1 -- 172.21.15.6:0/3574142087 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bc95401040 con 0x55bc955069a0 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc7faffd700 10 client.25859 handle_command_reply: tid=0 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 client.25859 shutdown 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 msgr2=0x55bc954fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc80c084300 tx=0x7fc80c09a460 comp rx=0 tx=0).stop 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 shutdown_connections 2023-11-17T00:10:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc955069a0 0x55bc95508db0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc954f3990 0x55bc954f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc953496b0 0x55bc954fc950 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc7faffd700 0 client.25859 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/3574142087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.685+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 >> 172.21.15.6:0/3574142087 conn(0x55bc954e31a0 msgr2=0x55bc954e7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.686+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 shutdown_connections 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.686+0000 7fc8359ffb80 1 -- 172.21.15.6:0/3574142087 wait complete. 2023-11-17T00:10:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.686+0000 7fc8359ffb80 20 client.25859 trim_cache size 0 max 16384 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.710+0000 7fc8359ffb80 1 Processor -- start 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.710+0000 7fc8359ffb80 1 -- start start 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc954f1850 con 0x55bc953496b0 2023-11-17T00:10:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc954f15f0 con 0x55bc954f3990 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc95509410 con 0x55bc954edcf0 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52520/0 (socket says 172.21.15.6:52520) 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81b7fe700 1 -- 172.21.15.6:0/372848063 learned_addr learned my addr 172.21.15.6:0/372848063 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81affd700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81bfff700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81b7fe700 1 -- 172.21.15.6:0/372848063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 msgr2=0x55bc954e8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.711+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954e8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc81b7fe700 1 -- 172.21.15.6:0/372848063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 msgr2=0x55bc954f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc81b7fe700 1 -- 172.21.15.6:0/372848063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc95404e80 con 0x55bc953496b0 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc81affd700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc80c09ab40 tx=0x7fc80c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7294d1e4466e5754 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc8027fc700 1 -- 172.21.15.6:0/372848063 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c098f50 con 0x55bc953496b0 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc8027fc700 1 -- 172.21.15.6:0/372848063 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc954e0960 con 0x55bc953496b0 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.712+0000 7fc8027fc700 1 -- 172.21.15.6:0/372848063 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0cb550 con 0x55bc953496b0 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- 172.21.15.6:0/372848063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 msgr2=0x55bc954f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc80c09ab40 tx=0x7fc80c005c00 comp rx=0 tx=0).stop 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- 172.21.15.6:0/372848063 shutdown_connections 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954f2c70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/372848063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- 172.21.15.6:0/372848063 >> 172.21.15.6:0/372848063 conn(0x55bc954e31a0 msgr2=0x55bc954ee0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- 172.21.15.6:0/372848063 shutdown_connections 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- 172.21.15.6:0/372848063 wait complete. 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.713+0000 7fc8359ffb80 1 -- start start 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 Processor -- start 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc954fb310 con 0x55bc953496b0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc95559820 con 0x55bc954f3990 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc95559960 con 0x55bc954edcf0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc8359ffb80 10 client.0 _ll_register_callbacks cb 0x7fc8100099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52526/0 (socket says 172.21.15.6:52526) 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc81b7fe700 1 -- 172.21.15.6:0/942378568 learned_addr learned my addr 172.21.15.6:0/942378568 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc81bfff700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.714+0000 7fc81affd700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 -- 172.21.15.6:0/942378568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 msgr2=0x55bc954fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 -- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 msgr2=0x55bc954fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 -- 172.21.15.6:0/942378568 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc810009e00 con 0x55bc953496b0 2023-11-17T00:10:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc81b7fe700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc80c09a8b0 tx=0x7fc80c09ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4783633267bed40d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc7faffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0cb750 con 0x55bc953496b0 2023-11-17T00:10:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc80c09ee40 con 0x55bc953496b0 2023-11-17T00:10:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.715+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc80c0a2f70 con 0x55bc953496b0 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80c099ca0 con 0x55bc953496b0 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 10 client.15804 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bc95405130 con 0x55bc953496b0 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80c0a2f70 con 0x55bc953496b0 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc80c0c1070 con 0x55bc953496b0 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 10 client.15804 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 20 client.15804 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc9555ab90 0x55bc9555cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 4 client.15804 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc9540d600 con 0x55bc9555ab90 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.716+0000 7fc81affd700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc9555ab90 0x55bc9555cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.717+0000 7fc81affd700 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc9555ab90 0x55bc9555cf70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308d5be75ee4bc85 server_cookie=6d650275cf76d091 in_seq=0 out_seq=0 2023-11-17T00:10:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.717+0000 7fc7faffd700 10 client.15804 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc7faffd700 1 -- 172.21.15.6:0/942378568 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc9540d600 con 0x55bc9555ab90 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc7faffd700 10 client.15804 handle_command_reply: tid=0 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc8359ffb80 1 client.15804 shutdown 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 msgr2=0x55bc954fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc80c09a8b0 tx=0x7fc80c09ace0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 shutdown_connections 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.735+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bc9555ab90 0x55bc9555cf70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bc953496b0 0x55bc954fce80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc7faffd700 0 client.15804 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bc954f3990 0x55bc954fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 --2- 172.21.15.6:0/942378568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc954edcf0 0x55bc954fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 >> 172.21.15.6:0/942378568 conn(0x55bc954e31a0 msgr2=0x55bc954f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 shutdown_connections 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 1 -- 172.21.15.6:0/942378568 wait complete. 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc8359ffb80 20 client.15804 trim_cache size 0 max 16384 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:10:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 msgr2=0x7fc8040a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.736+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 0x7fc8040a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814009100 comp rx=0 tx=0).stop 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 msgr2=0x7fc81c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b580 tx=0x7fc80c00f690 comp rx=0 tx=0).stop 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 shutdown_connections 2023-11-17T00:10:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8040a13e0 0x7fc8040a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc81c1002e0 0x7fc81c1983e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc81c105320 0x7fc81c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 --2- 172.21.15.6:0/325428396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc81c100c80 0x7fc81c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 >> 172.21.15.6:0/325428396 conn(0x7fc81c0fbb70 msgr2=0x7fc81c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 shutdown_connections 2023-11-17T00:10:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:20.737+0000 7fc820d9d700 1 -- 172.21.15.6:0/325428396 wait complete. 2023-11-17T00:10:20.927 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:20.927 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:20.931 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:20.931 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:20.957 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:20.957 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:20.957 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 39 -lt 100 ']' 2023-11-17T00:10:20.957 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 39 + 1 2023-11-17T00:10:20.984 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=40 2023-11-17T00:10:20.985 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:20.992 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:20.993 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:20.995 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:20.996 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:20.996 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:20 smithi087 ceph-mon[131182]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:20 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:10:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:20 smithi186 ceph-mon[131487]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:20 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:10:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:20 smithi006 ceph-mon[97840]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:20 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:10:21.420 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:21.420 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:21.565 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:21.730 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:21.731 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:21.731 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 40 -lt 100 ']' 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 40 + 1 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=41 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:21.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:22.261 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:22.261 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:22.499 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:22.499 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:22.508 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:22.508 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:22.509 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 41 -lt 100 ']' 2023-11-17T00:10:22.509 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 41 + 1 2023-11-17T00:10:22.552 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=42 2023-11-17T00:10:22.552 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:22.618 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:22.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:22.621 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:22.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:22.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:22 smithi087 ceph-mon[131182]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:22 smithi186 ceph-mon[131487]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:23.202 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:22 smithi006 ceph-mon[97840]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:23.353 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:23.353 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:23.504 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:23.504 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:23.524 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:23.524 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:23.524 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 42 -lt 100 ']' 2023-11-17T00:10:23.524 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 42 + 1 2023-11-17T00:10:23.576 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=43 2023-11-17T00:10:23.576 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:23.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:23.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:23.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:23.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:23.646 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 -- 172.21.15.6:0/2642188088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5240746a0 msgr2=0x7fa524072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2642188088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5240746a0 0x7fa524072d20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa514009990 tx=0x7fa514009ca0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 -- 172.21.15.6:0/2642188088 shutdown_connections 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2642188088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5240746a0 0x7fa524072d20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2642188088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2642188088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa524074b00 0x7fa524073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 -- 172.21.15.6:0/2642188088 >> 172.21.15.6:0/2642188088 conn(0x7fa5240774d0 msgr2=0x7fa5240778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:23.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 -- 172.21.15.6:0/2642188088 shutdown_connections 2023-11-17T00:10:23.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.882+0000 7fa5293ff700 1 -- 172.21.15.6:0/2642188088 wait complete. 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.883+0000 7fa5293ff700 1 Processor -- start 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.883+0000 7fa5293ff700 1 -- start start 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5240746a0 0x7fa524198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419d620 con 0x7fa524074b00 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419d790 con 0x7fa5240746a0 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5293ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419d950 con 0x7fa52410cec0 2023-11-17T00:10:23.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5237fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52540/0 (socket says 172.21.15.6:52540) 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 -- 172.21.15.6:0/2671215295 learned_addr learned my addr 172.21.15.6:0/2671215295 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa522ffd700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5240746a0 0x7fa524198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 -- 172.21.15.6:0/2671215295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 msgr2=0x7fa52419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 -- 172.21.15.6:0/2671215295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5240746a0 msgr2=0x7fa524198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5240746a0 0x7fa524198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5227fc700 1 -- 172.21.15.6:0/2671215295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140095f0 con 0x7fa524074b00 2023-11-17T00:10:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.884+0000 7fa5237fe700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa5227fc700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa51800d980 tx=0x7fa51800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518004220 con 0x7fa524074b00 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5241125a0 con 0x7fa524074b00 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa518007640 con 0x7fa524074b00 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518005690 con 0x7fa524074b00 2023-11-17T00:10:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.885+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa524112410 con 0x7fa524074b00 2023-11-17T00:10:23.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.886+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524112730 con 0x7fa524074b00 2023-11-17T00:10:23.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.888+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 25) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fa518004750 con 0x7fa524074b00 2023-11-17T00:10:23.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.890+0000 7fa50bfff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 0x7fa50c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:23.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.890+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa5180cd150 con 0x7fa524074b00 2023-11-17T00:10:23.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.890+0000 7fa522ffd700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 0x7fa50c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:23.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.891+0000 7fa522ffd700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 0x7fa50c0a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa51400b2e0 tx=0x7fa514005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:23.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:23.897+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa518095f60 con 0x7fa524074b00 2023-11-17T00:10:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.157+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa524199990 con 0x7fa524074b00 2023-11-17T00:10:24.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.158+0000 7fa50bfff700 1 -- 172.21.15.6:0/2671215295 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7fa518016090 con 0x7fa524074b00 2023-11-17T00:10:24.158 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:10:24.159 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:10:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 msgr2=0x7fa50c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 0x7fa50c0a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa51400b2e0 tx=0x7fa514005fb0 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 msgr2=0x7fa524198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa51800d980 tx=0x7fa51800dc90 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 shutdown_connections 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa50c0a13e0 0x7fa50c0a3890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa52410cec0 0x7fa52419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5240746a0 0x7fa524198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 --2- 172.21.15.6:0/2671215295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa524074b00 0x7fa524198b80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 >> 172.21.15.6:0/2671215295 conn(0x7fa5240774d0 msgr2=0x7fa524104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.160+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 shutdown_connections 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:24.161+0000 7fa5293ff700 1 -- 172.21.15.6:0/2671215295 wait complete. 2023-11-17T00:10:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:10:24.253 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:24.253 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:24.260 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:10:24.373 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:24.373 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:24.405 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:24.405 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:24.405 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 43 -lt 100 ']' 2023-11-17T00:10:24.406 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 43 + 1 2023-11-17T00:10:24.445 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=44 2023-11-17T00:10:24.446 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:24.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:24.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:24.463 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:24.464 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:24.464 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:24.952 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:24.953 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:25.065 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:25.066 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:25.091 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:25.091 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:25.091 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 44 -lt 100 ']' 2023-11-17T00:10:25.092 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 44 + 1 2023-11-17T00:10:25.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:24 smithi186 ceph-mon[131487]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:25.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:24 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2671215295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:25.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:24 smithi087 ceph-mon[131182]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:25.278 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:24 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2671215295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=45 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:25.279 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:25.279 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:24 smithi006 ceph-mon[97840]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 5 op/s 2023-11-17T00:10:25.279 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:24 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2671215295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:25.835 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:35.881 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:25 smithi087 ceph-mon[131182]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-11-17T00:10:35.881 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:26 smithi087 ceph-mon[131182]: pgmap v49: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:28 smithi087 ceph-mon[131182]: pgmap v50: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:30 smithi087 ceph-mon[131182]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:32 smithi087 ceph-mon[131182]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:34 smithi087 ceph-mon[131182]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:25 smithi186 ceph-mon[131487]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:26 smithi186 ceph-mon[131487]: pgmap v49: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:28 smithi186 ceph-mon[131487]: pgmap v50: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:30 smithi186 ceph-mon[131487]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:32 smithi186 ceph-mon[131487]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:35.882 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:34 smithi186 ceph-mon[131487]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 45 -lt 100 ']' 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 45 + 1 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=46 2023-11-17T00:10:35.883 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:35.884 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:35.884 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:35.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:35.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:35.884 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:35.884 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:25 smithi006 ceph-mon[97840]: mgrmap e26: x(active, since 91s), standbys: z, y 2023-11-17T00:10:35.933 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:35.973 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:35.973 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:35.973 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:35.973 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:35.974 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:35.974 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 46 -lt 100 ']' 2023-11-17T00:10:35.974 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 46 + 1 2023-11-17T00:10:35.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 -- 172.21.15.6:0/1421253043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c010ce50 msgr2=0x7f60c010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 --2- 172.21.15.6:0/1421253043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c010ce50 0x7f60c010f230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f60b800b0a0 tx=0x7f60b800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 -- 172.21.15.6:0/1421253043 shutdown_connections 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 --2- 172.21.15.6:0/1421253043 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c010ce50 0x7f60c010f230 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 --2- 172.21.15.6:0/1421253043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 --2- 172.21.15.6:0/1421253043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c0074b00 0x7f60c0073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 -- 172.21.15.6:0/1421253043 >> 172.21.15.6:0/1421253043 conn(0x7f60c0077500 msgr2=0x7f60c0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 -- 172.21.15.6:0/1421253043 shutdown_connections 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.089+0000 7f60c51ff700 1 -- 172.21.15.6:0/1421253043 wait complete. 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.090+0000 7f60c51ff700 1 Processor -- start 2023-11-17T00:10:35.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 -- start start 2023-11-17T00:10:36.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c010ce50 0x7f60c019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019d650 con 0x7f60c0074b00 2023-11-17T00:10:36.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019d7c0 con 0x7f60c00746a0 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60c51ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019d980 con 0x7f60c010ce50 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60bed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52568/0 (socket says 172.21.15.6:52568) 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60be59c700 1 -- 172.21.15.6:0/2453728953 learned_addr learned my addr 172.21.15.6:0/2453728953 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.091+0000 7f60bf59e700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c010ce50 0x7f60c019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 -- 172.21.15.6:0/2453728953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c010ce50 msgr2=0x7f60c019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c010ce50 0x7f60c019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 -- 172.21.15.6:0/2453728953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 msgr2=0x7f60c0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 -- 172.21.15.6:0/2453728953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009d00 con 0x7f60c0074b00 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60be59c700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9a0 tx=0x7f60b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60a7fff700 1 -- 172.21.15.6:0/2453728953 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4009790 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60a7fff700 1 -- 172.21.15.6:0/2453728953 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b4010430 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60a7fff700 1 -- 172.21.15.6:0/2453728953 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4018680 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c0112520 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.092+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c0112390 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.095+0000 7f60a7fff700 1 -- 172.21.15.6:0/2453728953 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f60b40105a0 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.097+0000 7f60a7fff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 0x7f60a80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.097+0000 7f60a7fff700 1 -- 172.21.15.6:0/2453728953 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60b4014070 con 0x7f60c0074b00 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.097+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 0x7f60a80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.098+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 0x7f60a80a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f60b0011490 tx=0x7f60b0009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.076 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=47 2023-11-17T00:10:36.076 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.076 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.076 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 Processor -- start 2023-11-17T00:10:36.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 -- start start 2023-11-17T00:10:36.077 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6946b60 0x5629d6946f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d694b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d684c060 con 0x5629d6946b60 2023-11-17T00:10:36.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d689e8a0 con 0x5629d694b8b0 2023-11-17T00:10:36.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6897d30 con 0x5629d6947470 2023-11-17T00:10:36.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60bed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6946b60 0x5629d6946f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.191+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:57734/0 (socket says 172.21.15.6:57734) 2023-11-17T00:10:36.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 -- 172.21.15.6:0/115102617 learned_addr learned my addr 172.21.15.6:0/115102617 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60be59c700 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d694b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.099 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 -- 172.21.15.6:0/115102617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 msgr2=0x5629d694b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d694b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 -- 172.21.15.6:0/115102617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6946b60 msgr2=0x5629d6946f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6946b60 0x5629d6946f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 -- 172.21.15.6:0/115102617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d6719630 con 0x5629d694b8b0 2023-11-17T00:10:36.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60bf59e700 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60b800b0a0 tx=0x7f60b8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf8fc4eff15ba2e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.100 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.192+0000 7f60a57fa700 1 -- 172.21.15.6:0/115102617 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800e040 con 0x5629d694b8b0 2023-11-17T00:10:36.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60a57fa700 1 -- 172.21.15.6:0/115102617 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b800bcb0 con 0x5629d694b8b0 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60a57fa700 1 -- 172.21.15.6:0/115102617 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b801c640 con 0x5629d694b8b0 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 -- 172.21.15.6:0/115102617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 msgr2=0x5629d694bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60b800b0a0 tx=0x7f60b8009c20 comp rx=0 tx=0).stop 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 -- 172.21.15.6:0/115102617 shutdown_connections 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6946b60 0x5629d6946f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d694bd00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/115102617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d694b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.193+0000 7f60d8aecb80 1 -- 172.21.15.6:0/115102617 >> 172.21.15.6:0/115102617 conn(0x5629d693c360 msgr2=0x5629d6941b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.194+0000 7f60d8aecb80 1 -- 172.21.15.6:0/115102617 shutdown_connections 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.194+0000 7f60d8aecb80 1 -- 172.21.15.6:0/115102617 wait complete. 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.194+0000 7f60d8aecb80 1 -- start start 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.194+0000 7f60d8aecb80 1 Processor -- start 2023-11-17T00:10:36.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d6953030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6947470 0x5629d6950240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d68398e0 con 0x5629d694b8b0 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6839580 con 0x5629d6947470 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d671b920 con 0x5629d6946b60 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60d8aecb80 10 client.0 _ll_register_callbacks cb 0x5629d6857780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52584/0 (socket says 172.21.15.6:52584) 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60bf59e700 1 -- 172.21.15.6:0/3157111450 learned_addr learned my addr 172.21.15.6:0/3157111450 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60bed9d700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d6953030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.195+0000 7f60be59c700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6947470 0x5629d6950240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 -- 172.21.15.6:0/3157111450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 msgr2=0x5629d6953030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d6953030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 -- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6947470 msgr2=0x5629d6950240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6947470 0x5629d6950240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 -- 172.21.15.6:0/3157111450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c010fb20 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60bf59e700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f60b800b3f0 tx=0x7f60b8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35dc7c276579e9c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800e040 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629d6939b20 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.196+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60b800be20 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b80074a0 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 10 client.15822 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5629d685e190 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629d6939b20 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60b8019030 con 0x5629d694b8b0 2023-11-17T00:10:36.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 20 client.15822 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69602c0 0x5629d69626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 4 client.15822 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629d685a0a0 con 0x5629d69602c0 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.197+0000 7f60bed9d700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69602c0 0x5629d69626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.198+0000 7f60a2ff5700 10 client.15822 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.198+0000 7f60bed9d700 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69602c0 0x5629d69626a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371abc541c1a281a server_cookie=1f144a215dccf3b7 in_seq=0 out_seq=0 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60a2ff5700 1 -- 172.21.15.6:0/3157111450 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629d685a0a0 con 0x5629d69602c0 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60a2ff5700 10 client.15822 handle_command_reply: tid=0 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 client.15822 shutdown 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 msgr2=0x5629d6950830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f60b800b3f0 tx=0x7f60b8009590 comp rx=0 tx=0).stop 2023-11-17T00:10:36.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 shutdown_connections 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69602c0 0x5629d69626a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6950830 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d6953030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/3157111450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6947470 0x5629d6950240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 >> 172.21.15.6:0/3157111450 conn(0x5629d693c360 msgr2=0x5629d694cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60a2ff5700 0 client.15822 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 shutdown_connections 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.199+0000 7f60d8aecb80 1 -- 172.21.15.6:0/3157111450 wait complete. 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.200+0000 7f60d8aecb80 20 client.15822 trim_cache size 0 max 16384 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 Processor -- start 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 -- start start 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6946b60 0x5629d6952b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d69498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6947840 con 0x5629d694b8b0 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d694a360 con 0x5629d6946b60 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.213+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6939960 con 0x5629d6947470 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d69498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60bf59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52598/0 (socket says 172.21.15.6:52598) 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60bf59e700 1 -- 172.21.15.6:0/2252278076 learned_addr learned my addr 172.21.15.6:0/2252278076 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6946b60 0x5629d6952b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 -- 172.21.15.6:0/2252278076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6946b60 msgr2=0x5629d6952b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6946b60 0x5629d6952b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 -- 172.21.15.6:0/2252278076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 msgr2=0x5629d6941560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 -- 172.21.15.6:0/2252278076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d685dee0 con 0x5629d6947470 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60bf59e700 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.214+0000 7f60be59c700 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d69498b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ca590 tx=0x7f60b4096380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959b66c7c48672d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60a5ffb700 1 -- 172.21.15.6:0/2252278076 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40b6070 con 0x5629d6947470 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60a5ffb700 1 -- 172.21.15.6:0/2252278076 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b409df40 con 0x5629d6947470 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60a5ffb700 1 -- 172.21.15.6:0/2252278076 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4099400 con 0x5629d6947470 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2252278076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 msgr2=0x5629d69498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d69498b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ca590 tx=0x7f60b4096380 comp rx=0 tx=0).stop 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2252278076 shutdown_connections 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d694b8b0 0x5629d6941560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d6946b60 0x5629d6952b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2252278076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6947470 0x5629d69498b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2252278076 >> 172.21.15.6:0/2252278076 conn(0x5629d693c360 msgr2=0x5629d695cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2252278076 shutdown_connections 2023-11-17T00:10:36.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.215+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2252278076 wait complete. 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.216+0000 7f60d8aecb80 1 -- start start 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.216+0000 7f60d8aecb80 1 Processor -- start 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.216+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d69a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.216+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60d8aecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d69a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d69b3250 con 0x5629d6947470 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d69b3390 con 0x5629d694b8b0 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60d8aecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d69b3550 con 0x5629d6946b60 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60d8aecb80 10 client.0 _ll_register_callbacks cb 0x7f60b0004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52600/0 (socket says 172.21.15.6:52600) 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 -- 172.21.15.6:0/2050223476 learned_addr learned my addr 172.21.15.6:0/2050223476 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60bf59e700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d69a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d69a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 -- 172.21.15.6:0/2050223476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 msgr2=0x5629d69a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d69a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 -- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 msgr2=0x5629d69a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d69a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 -- 172.21.15.6:0/2050223476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629d685e190 con 0x5629d6947470 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60bf59e700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d69a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60be59c700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ca5f0 tx=0x7f60b4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82789b73ee6b5b42 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.217+0000 7f60a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40b6070 con 0x5629d6947470 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60b409a170 con 0x5629d6947470 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60b800bcb0 con 0x5629d6947470 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b409a330 con 0x5629d6947470 2023-11-17T00:10:36.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 10 client.15828 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f60c0112390 con 0x5629d6947470 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b800bcb0 con 0x5629d6947470 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60b4098e70 con 0x5629d6947470 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 10 client.15828 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 20 client.15828 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69b4870 0x5629d69b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 4 client.15828 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.218+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629d6866660 con 0x5629d69b4870 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69b4870 0x5629d69b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60bed9d700 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69b4870 0x5629d69b6c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12240d51b166f0b server_cookie=3a5db5a1d30925ff in_seq=0 out_seq=0 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60a2ff5700 10 client.15828 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60a2ff5700 1 -- 172.21.15.6:0/2050223476 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629d6866660 con 0x5629d69b4870 2023-11-17T00:10:36.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60a2ff5700 10 client.15828 handle_command_reply: tid=0 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.219+0000 7f60d8aecb80 1 client.15828 shutdown 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 msgr2=0x5629d69a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f60b40ca5f0 tx=0x7f60b4010c80 comp rx=0 tx=0).stop 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 shutdown_connections 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5629d694b8b0 0x5629d69a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629d6946b60 0x5629d69a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5629d69b4870 0x5629d69b6c80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 --2- 172.21.15.6:0/2050223476 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5629d6947470 0x5629d69a85e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60a2ff5700 0 client.15828 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 >> 172.21.15.6:0/2050223476 conn(0x5629d693c360 msgr2=0x5629d69498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 shutdown_connections 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 1 -- 172.21.15.6:0/2050223476 wait complete. 2023-11-17T00:10:36.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60d8aecb80 20 client.15828 trim_cache size 0 max 16384 2023-11-17T00:10:36.120 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:10:36.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 msgr2=0x7f60a80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.220+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 0x7f60a80a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f60b0011490 tx=0x7f60b0009260 comp rx=0 tx=0).stop 2023-11-17T00:10:36.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 msgr2=0x7f60c0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e9a0 tx=0x7f60b400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 shutdown_connections 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f60a80a1340 0x7f60a80a37f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60c010ce50 0x7f60c019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60c00746a0 0x7f60c0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 --2- 172.21.15.6:0/2453728953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60c0074b00 0x7f60c0198bb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 >> 172.21.15.6:0/2453728953 conn(0x7f60c0077500 msgr2=0x7f60c0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.121 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:26 smithi006 ceph-mon[97840]: pgmap v49: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-11-17T00:10:36.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 shutdown_connections 2023-11-17T00:10:36.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:27.221+0000 7f60c51ff700 1 -- 172.21.15.6:0/2453728953 wait complete. 2023-11-17T00:10:36.128 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.128 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 47 -lt 100 ']' 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 47 + 1 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=48 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.129 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 48 -lt 100 ']' 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 48 + 1 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=49 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 49 -lt 100 ']' 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 49 + 1 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=50 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 50 -lt 100 ']' 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 50 + 1 2023-11-17T00:10:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=51 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.133 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 51 -lt 100 ']' 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 51 + 1 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=52 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.134 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 52 -lt 100 ']' 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 52 + 1 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=53 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.136 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:28 smithi006 ceph-mon[97840]: pgmap v50: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-17T00:10:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 53 -lt 100 ']' 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 53 + 1 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=54 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 54 -lt 100 ']' 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 54 + 1 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=55 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.142 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 55 -lt 100 ']' 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 55 + 1 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=56 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 56 -lt 100 ']' 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 56 + 1 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=57 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.144 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.145 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.145 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.145 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:30 smithi006 ceph-mon[97840]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-11-17T00:10:36.149 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.149 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 57 -lt 100 ']' 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 57 + 1 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=58 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 58 -lt 100 ']' 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 58 + 1 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=59 2023-11-17T00:10:36.151 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 59 -lt 100 ']' 2023-11-17T00:10:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 59 + 1 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=60 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.153 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 60 -lt 100 ']' 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 60 + 1 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=61 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.154 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:32 smithi006 ceph-mon[97840]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 61 -lt 100 ']' 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 61 + 1 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=62 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 62 -lt 100 ']' 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 62 + 1 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=63 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.159 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.159 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.159 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.159 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:34 smithi006 ceph-mon[97840]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-17T00:10:36.160 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.160 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.160 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.160 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 63 -lt 100 ']' 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 63 + 1 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=64 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 -- 172.21.15.6:0/484104510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 msgr2=0x7f9eb410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/484104510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb410d0b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00b0d0 tx=0x7f9eac00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 -- 172.21.15.6:0/484104510 shutdown_connections 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/484104510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb410d0b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/484104510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb41004a0 0x7f9eb4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/484104510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb4100e40 0x7f9eb4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 -- 172.21.15.6:0/484104510 >> 172.21.15.6:0/484104510 conn(0x7f9eb40fbd10 msgr2=0x7f9eb40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 -- 172.21.15.6:0/484104510 shutdown_connections 2023-11-17T00:10:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.355+0000 7f9ebbdff700 1 -- 172.21.15.6:0/484104510 wait complete. 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.357+0000 7f9ebbdff700 1 Processor -- start 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.357+0000 7f9ebbdff700 1 -- start start 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb41004a0 0x7f9eb4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb4100e40 0x7f9eb4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419c600 con 0x7f9eb41054e0 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419c770 con 0x7f9eb4100e40 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9ebbdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419c930 con 0x7f9eb41004a0 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34146/0 (socket says 172.21.15.6:34146) 2023-11-17T00:10:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb41004a0 0x7f9eb4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 -- 172.21.15.6:0/2608910085 learned_addr learned my addr 172.21.15.6:0/2608910085 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eb939a700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb4100e40 0x7f9eb4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 -- 172.21.15.6:0/2608910085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb41004a0 msgr2=0x7f9eb4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb41004a0 0x7f9eb4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 -- 172.21.15.6:0/2608910085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb4100e40 msgr2=0x7f9eb4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.358+0000 7f9eba39c700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb4100e40 0x7f9eb4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9eba39c700 1 -- 172.21.15.6:0/2608910085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0009520 con 0x7f9eb41054e0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9eba39c700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9eac007820 tx=0x7f9eac007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9eaaffd700 1 -- 172.21.15.6:0/2608910085 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00e040 con 0x7f9eb41054e0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9eaaffd700 1 -- 172.21.15.6:0/2608910085 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eac00bcb0 con 0x7f9eb41054e0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac009d00 con 0x7f9eb41054e0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9eaaffd700 1 -- 172.21.15.6:0/2608910085 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0058a0 con 0x7f9eb41054e0 2023-11-17T00:10:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.359+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb4110300 con 0x7f9eb41054e0 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.362+0000 7f9eaaffd700 1 -- 172.21.15.6:0/2608910085 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9eac004a50 con 0x7f9eb41054e0 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.364+0000 7f9eaaffd700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 0x7f9ea00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.364+0000 7f9eaaffd700 1 -- 172.21.15.6:0/2608910085 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9eac0ce100 con 0x7f9eb41054e0 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.364+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 0x7f9ea00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.365+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 0x7f9ea00a37f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009df0 tx=0x7f9eb0009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.416+0000 7f9ecf731b80 1 Processor -- start 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.416+0000 7f9ecf731b80 1 -- start start 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 0x560c5cce9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cbef000 con 0x560c5ccef720 2023-11-17T00:10:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cc41840 con 0x560c5cb456b0 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cc3acd0 con 0x560c5ccefb90 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9eb939a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9eb939a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34148/0 (socket says 172.21.15.6:34148) 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9eb939a700 1 -- 172.21.15.6:0/1684807852 learned_addr learned my addr 172.21.15.6:0/1684807852 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.417+0000 7f9eba39c700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 0x560c5cce9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 -- 172.21.15.6:0/1684807852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 msgr2=0x560c5cce9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 0x560c5cce9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 -- 172.21.15.6:0/1684807852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 msgr2=0x560c5ccef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 -- 172.21.15.6:0/1684807852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5cabc5d0 con 0x560c5ccef720 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eba39c700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 0x560c5cce9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009cb0 tx=0x7f9ea40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81a56d3120ed235 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9e9ffff700 1 -- 172.21.15.6:0/1684807852 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4011ce0 con 0x560c5ccef720 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9e9ffff700 1 -- 172.21.15.6:0/1684807852 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea4011e40 con 0x560c5ccef720 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.418+0000 7f9e9ffff700 1 -- 172.21.15.6:0/1684807852 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4005280 con 0x560c5ccef720 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1684807852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 msgr2=0x560c5cce9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009cb0 tx=0x7f9ea40058e0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1684807852 shutdown_connections 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccefb90 0x560c5cce9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1684807852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccef720 0x560c5cce9560 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1684807852 >> 172.21.15.6:0/1684807852 conn(0x560c5ccdef50 msgr2=0x560c5cce4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1684807852 shutdown_connections 2023-11-17T00:10:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.419+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1684807852 wait complete. 2023-11-17T00:10:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.443+0000 7f9ecf731b80 1 -- start start 2023-11-17T00:10:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.443+0000 7f9ecf731b80 1 Processor -- start 2023-11-17T00:10:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccef720 0x560c5ccf2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cbdc880 con 0x560c5ccefb90 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cbdc520 con 0x560c5ccef720 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cabe8c0 con 0x560c5cb456b0 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9ecf731b80 10 client.0 _ll_register_callbacks cb 0x560c5cbfa720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38172/0 (socket says 172.21.15.6:38172) 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/465004690 learned_addr learned my addr 172.21.15.6:0/465004690 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eb939a700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccef720 0x560c5ccf2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 -- 172.21.15.6:0/465004690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 msgr2=0x560c5ccf5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 -- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccef720 msgr2=0x560c5ccf2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccef720 0x560c5ccf2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.463+0000 7f9eba39c700 1 -- 172.21.15.6:0/465004690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea40098e0 con 0x560c5ccefb90 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9eba39c700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf34e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cb4a0 tx=0x7f9eac09a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74baada9450f20c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9e9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0b2070 con 0x560c5ccefb90 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9eac09aeb0 con 0x560c5ccefb90 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c5ccdc710 con 0x560c5ccefb90 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eac09eee0 con 0x560c5ccefb90 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9ecf731b80 10 client.15840 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560c5cc01130 con 0x560c5ccefb90 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.464+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eac0c9020 con 0x560c5ccefb90 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9eac09b110 con 0x560c5ccefb90 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 20 client.15840 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd02e50 0x560c5cd05230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 4 client.15840 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c5cbfd040 con 0x560c5cd02e50 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.465+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd02e50 0x560c5cd05230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.466+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd02e50 0x560c5cd05230 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9573e85983695b37 server_cookie=cfb935cad213d33e in_seq=0 out_seq=0 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.466+0000 7f9e9d7fa700 10 client.15840 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.466+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/465004690 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c5cbfd040 con 0x560c5cd02e50 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.466+0000 7f9e9d7fa700 10 client.15840 handle_command_reply: tid=0 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 client.15840 shutdown 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 msgr2=0x560c5ccf34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf34e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cb4a0 tx=0x7f9eac09a9f0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 shutdown_connections 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd02e50 0x560c5cd05230 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf34e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5cb456b0 0x560c5ccf5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9e9d7fa700 0 client.15840 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/465004690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccef720 0x560c5ccf2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 >> 172.21.15.6:0/465004690 conn(0x560c5ccdef50 msgr2=0x560c5cce37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 shutdown_connections 2023-11-17T00:10:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 1 -- 172.21.15.6:0/465004690 wait complete. 2023-11-17T00:10:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.467+0000 7f9ecf731b80 20 client.15840 trim_cache size 0 max 16384 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 Processor -- start 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 -- start start 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccefb90 0x560c5cce3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cce1020 con 0x560c5cb456b0 2023-11-17T00:10:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5ccf4340 con 0x560c5ccefb90 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cce53c0 con 0x560c5ccef720 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34160/0 (socket says 172.21.15.6:34160) 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/2214435833 learned_addr learned my addr 172.21.15.6:0/2214435833 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.483+0000 7f9eba39c700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccefb90 0x560c5cce3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb939a700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/2214435833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 msgr2=0x560c5ccf5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/2214435833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccefb90 msgr2=0x560c5cce3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccefb90 0x560c5cce3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/2214435833 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5cce06f0 con 0x560c5cb456b0 2023-11-17T00:10:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb939a700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:10:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0013010 tx=0x7f9eb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad21e68c64eefb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9ea8bff700 1 -- 172.21.15.6:0/2214435833 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0022070 con 0x560c5cb456b0 2023-11-17T00:10:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9ea8bff700 1 -- 172.21.15.6:0/2214435833 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eb0023040 con 0x560c5cb456b0 2023-11-17T00:10:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.484+0000 7f9ea8bff700 1 -- 172.21.15.6:0/2214435833 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000ea00 con 0x560c5cb456b0 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- 172.21.15.6:0/2214435833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 msgr2=0x560c5ccfa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0013010 tx=0x7f9eb0014000 comp rx=0 tx=0).stop 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- 172.21.15.6:0/2214435833 shutdown_connections 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5cb456b0 0x560c5ccfa070 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5ccefb90 0x560c5cce3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/2214435833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- 172.21.15.6:0/2214435833 >> 172.21.15.6:0/2214435833 conn(0x560c5ccdef50 msgr2=0x560c5ccfe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- 172.21.15.6:0/2214435833 shutdown_connections 2023-11-17T00:10:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- 172.21.15.6:0/2214435833 wait complete. 2023-11-17T00:10:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.485+0000 7f9ecf731b80 1 -- start start 2023-11-17T00:10:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 Processor -- start 2023-11-17T00:10:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cd55cd0 con 0x560c5ccefb90 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cd55e10 con 0x560c5cb456b0 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5cd55f80 con 0x560c5ccef720 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9ecf731b80 10 client.0 _ll_register_callbacks cb 0x7f9eac009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9eb9b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:37264/0 (socket says 172.21.15.6:37264) 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/1064544768 learned_addr learned my addr 172.21.15.6:0/1064544768 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.486+0000 7f9eba39c700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/1064544768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 msgr2=0x560c5ccf6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/1064544768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 msgr2=0x560c5ccf70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 -- 172.21.15.6:0/1064544768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb002bab0 con 0x560c5cb456b0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9eb9b9b700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009440 tx=0x7f9eb0003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18816ae44f1c2e57 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9e9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0022050 con 0x560c5cb456b0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9eb002b0b0 con 0x560c5cb456b0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.487+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ea4011e40 con 0x560c5cb456b0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb002b270 con 0x560c5cb456b0 2023-11-17T00:10:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 10 client.25907 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560c5ca68d80 con 0x560c5cb456b0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ea4011e40 con 0x560c5cb456b0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9eb000e4e0 con 0x560c5cb456b0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 20 client.25907 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd575a0 0x560c5cd599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 4 client.25907 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.488+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c5cc09600 con 0x560c5cd575a0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.489+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd575a0 0x560c5cd599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.489+0000 7f9eb939a700 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd575a0 0x560c5cd599b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6b4ae5bc581cd6 server_cookie=98cb58f4394f3c73 in_seq=0 out_seq=0 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.489+0000 7f9e9d7fa700 10 client.25907 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9e9d7fa700 1 -- 172.21.15.6:0/1064544768 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c5cc09600 con 0x560c5cd575a0 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9e9d7fa700 10 client.25907 handle_command_reply: tid=0 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 client.25907 shutdown 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 msgr2=0x560c5ccf6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009440 tx=0x7f9eb0003f90 comp rx=0 tx=0).stop 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 shutdown_connections 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x560c5cd575a0 0x560c5cd599b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560c5ccefb90 0x560c5ccf70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x560c5cb456b0 0x560c5ccf6650 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 --2- 172.21.15.6:0/1064544768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c5ccef720 0x560c5ccf6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9e9d7fa700 0 client.25907 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 >> 172.21.15.6:0/1064544768 conn(0x560c5ccdef50 msgr2=0x560c5ccecee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.511+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 shutdown_connections 2023-11-17T00:10:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ecf731b80 1 -- 172.21.15.6:0/1064544768 wait complete. 2023-11-17T00:10:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ecf731b80 20 client.25907 trim_cache size 0 max 16384 2023-11-17T00:10:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:10:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:10:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:10:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 msgr2=0x7f9ea00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 0x7f9ea00a37f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009df0 tx=0x7f9eb0009260 comp rx=0 tx=0).stop 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 msgr2=0x7f9eb419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9eac007820 tx=0x7f9eac007850 comp rx=0 tx=0).stop 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 shutdown_connections 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9ea00a1340 0x7f9ea00a37f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9eb41054e0 0x7f9eb419c000 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9eb41004a0 0x7f9eb4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 --2- 172.21.15.6:0/2608910085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9eb4100e40 0x7f9eb4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.512+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 >> 172.21.15.6:0/2608910085 conn(0x7f9eb40fbd10 msgr2=0x7f9eb410b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.513+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 shutdown_connections 2023-11-17T00:10:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:36.513+0000 7f9ebbdff700 1 -- 172.21.15.6:0/2608910085 wait complete. 2023-11-17T00:10:36.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:36.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:36.838 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.838 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:36.838 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 64 -lt 100 ']' 2023-11-17T00:10:36.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 64 + 1 2023-11-17T00:10:36.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=65 2023-11-17T00:10:36.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:36.904 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:36.905 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:36.905 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:36.905 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:36.906 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:37.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:10:37.124 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:10:37.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:36 smithi006 ceph-mon[97840]: pgmap v54: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-11-17T00:10:37.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:36 smithi087 ceph-mon[131182]: pgmap v54: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-11-17T00:10:37.361 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:37.362 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:37.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:36 smithi186 ceph-mon[131487]: pgmap v54: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-11-17T00:10:37.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:37.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:37.393 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:37.393 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:37.394 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 65 -lt 100 ']' 2023-11-17T00:10:37.394 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 65 + 1 2023-11-17T00:10:37.432 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=66 2023-11-17T00:10:37.433 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:37.441 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:37.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:37.443 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:37.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:37.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:37.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:37.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:37.955 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:37.956 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:37.980 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:37.980 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:37.980 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 66 -lt 100 ']' 2023-11-17T00:10:37.981 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 66 + 1 2023-11-17T00:10:38.032 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=67 2023-11-17T00:10:38.033 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:38.091 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:38.092 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:38.094 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:38.094 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:38.095 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:38.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:38.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:38.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:38.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:38.813 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:38.813 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:38.813 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 67 -lt 100 ']' 2023-11-17T00:10:38.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 67 + 1 2023-11-17T00:10:38.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=68 2023-11-17T00:10:38.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:38.921 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:38.921 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:38.925 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:38.925 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:38.925 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:38 smithi006 ceph-mon[97840]: pgmap v55: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-11-17T00:10:39.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:38 smithi087 ceph-mon[131182]: pgmap v55: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-11-17T00:10:39.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:38 smithi186 ceph-mon[131487]: pgmap v55: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 3 op/s 2023-11-17T00:10:39.526 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:39.526 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:39.702 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:39.702 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:39.721 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:39.722 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:39.722 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 68 -lt 100 ']' 2023-11-17T00:10:39.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 68 + 1 2023-11-17T00:10:39.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=69 2023-11-17T00:10:39.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:39.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:39.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:39.829 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:39.830 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:39.830 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:40.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.070+0000 7f71435ff700 1 -- 172.21.15.6:0/2252194067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 msgr2=0x7f713c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.070+0000 7f71435ff700 1 --2- 172.21.15.6:0/2252194067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c104e80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f712c0099c0 tx=0x7f712c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 -- 172.21.15.6:0/2252194067 shutdown_connections 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 --2- 172.21.15.6:0/2252194067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 0x7f713c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 --2- 172.21.15.6:0/2252194067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 0x7f713c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 --2- 172.21.15.6:0/2252194067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c104e80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 -- 172.21.15.6:0/2252194067 >> 172.21.15.6:0/2252194067 conn(0x7f713c077500 msgr2=0x7f713c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 -- 172.21.15.6:0/2252194067 shutdown_connections 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.071+0000 7f71435ff700 1 -- 172.21.15.6:0/2252194067 wait complete. 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 Processor -- start 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 -- start start 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 0x7f713c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 0x7f713c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c19b3f0 con 0x7f713c1015f0 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c19b560 con 0x7f713c105560 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.072+0000 7f71435ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c19b720 con 0x7f713c100c50 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51520/0 (socket says 172.21.15.6:51520) 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 -- 172.21.15.6:0/659701445 learned_addr learned my addr 172.21.15.6:0/659701445 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7141b9c700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 0x7f713c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f714139b700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 0x7f713c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 -- 172.21.15.6:0/659701445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 msgr2=0x7f713c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 0x7f713c196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 -- 172.21.15.6:0/659701445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 msgr2=0x7f713c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 0x7f713c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 -- 172.21.15.6:0/659701445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0095f0 con 0x7f713c1015f0 2023-11-17T00:10:40.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f7140b9a700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f712c009990 tx=0x7f712c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f71327fc700 1 -- 172.21.15.6:0/659701445 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c01d070 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f71327fc700 1 -- 172.21.15.6:0/659701445 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c005020 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.073+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c110320 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.074+0000 7f71327fc700 1 -- 172.21.15.6:0/659701445 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c004a70 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.074+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c110190 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.077+0000 7f71327fc700 1 -- 172.21.15.6:0/659701445 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f712c005400 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.078+0000 7f71327fc700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 0x7f71280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.078+0000 7f71327fc700 1 -- 172.21.15.6:0/659701445 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f712c0cda20 con 0x7f713c1015f0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.078+0000 7f714139b700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 0x7f71280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.079+0000 7f714139b700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 0x7f71280a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7138009c60 tx=0x7f7138009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.148+0000 7f7156fb6b80 1 Processor -- start 2023-11-17T00:10:40.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.148+0000 7f7156fb6b80 1 -- start start 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.148+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.148+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.148+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd20b10e0 con 0x562fd21b4d40 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd2103920 con 0x562fd21afff0 2023-11-17T00:10:40.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd20fcdb0 con 0x562fd21b0900 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51530/0 (socket says 172.21.15.6:51530) 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f714139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 -- 172.21.15.6:0/926399561 learned_addr learned my addr 172.21.15.6:0/926399561 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7140b9a700 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 -- 172.21.15.6:0/926399561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 msgr2=0x562fd21b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 -- 172.21.15.6:0/926399561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 msgr2=0x562fd21b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.149+0000 7f7141b9c700 1 -- 172.21.15.6:0/926399561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd1f7e6b0 con 0x562fd21b4d40 2023-11-17T00:10:40.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.150+0000 7f7141b9c700 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7134003f30 tx=0x7f7134009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f4771c71a16fb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.150+0000 7f71277fe700 1 -- 172.21.15.6:0/926399561 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134003a00 con 0x562fd21b4d40 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.150+0000 7f71277fe700 1 -- 172.21.15.6:0/926399561 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f713400f040 con 0x562fd21b4d40 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.150+0000 7f71277fe700 1 -- 172.21.15.6:0/926399561 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340156e0 con 0x562fd21b4d40 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- 172.21.15.6:0/926399561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 msgr2=0x562fd21b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7134003f30 tx=0x7f7134009c20 comp rx=0 tx=0).stop 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- 172.21.15.6:0/926399561 shutdown_connections 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b5190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/926399561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- 172.21.15.6:0/926399561 >> 172.21.15.6:0/926399561 conn(0x562fd21a13e0 msgr2=0x562fd21a6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- 172.21.15.6:0/926399561 shutdown_connections 2023-11-17T00:10:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- 172.21.15.6:0/926399561 wait complete. 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.151+0000 7f7156fb6b80 1 -- start start 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 Processor -- start 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd21bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd21b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd209e960 con 0x562fd21b4d40 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd209e600 con 0x562fd21b0900 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd1f809a0 con 0x562fd21afff0 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7156fb6b80 10 client.0 _ll_register_callbacks cb 0x562fd20bc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7141b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7141b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51542/0 (socket says 172.21.15.6:51542) 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7141b9c700 1 -- 172.21.15.6:0/3085209246 learned_addr learned my addr 172.21.15.6:0/3085209246 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:40.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.152+0000 7f7140b9a700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd21b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f714139b700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd21bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 -- 172.21.15.6:0/3085209246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 msgr2=0x562fd21bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd21bc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 -- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 msgr2=0x562fd21b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd21b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 -- 172.21.15.6:0/3085209246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7134003b60 con 0x562fd21b4d40 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7140b9a700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd21b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7141b9c700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7134005b20 tx=0x7f7134005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381188622ddc90c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7124ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:40.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134004880 con 0x562fd21b4d40 2023-11-17T00:10:40.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fd219eba0 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.153+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7134015480 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7134015640 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 10 client.15858 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562fd20c3210 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562fd219eba0 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7134025020 con 0x562fd21b4d40 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 20 client.15858 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd21c9ba0 0x562fd21cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 4 client.15858 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:40.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.154+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fd20bf120 con 0x562fd21c9ba0 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.155+0000 7f714139b700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd21c9ba0 0x562fd21cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.155+0000 7f714139b700 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd21c9ba0 0x562fd21cbfb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c033e0f81b214554 server_cookie=b0e0cdfb2a65be2d in_seq=0 out_seq=0 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.155+0000 7f7124ff9700 10 client.15858 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7124ff9700 1 -- 172.21.15.6:0/3085209246 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562fd20bf120 con 0x562fd21c9ba0 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7124ff9700 10 client.15858 handle_command_reply: tid=0 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 client.15858 shutdown 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 msgr2=0x562fd21b9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7134005b20 tx=0x7f7134005b50 comp rx=0 tx=0).stop 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 shutdown_connections 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd21c9ba0 0x562fd21cbfb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21b9d00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd21bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/3085209246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd21b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7124ff9700 0 client.15858 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:40.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 >> 172.21.15.6:0/3085209246 conn(0x562fd21a13e0 msgr2=0x562fd21b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.156+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 shutdown_connections 2023-11-17T00:10:40.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.157+0000 7f7156fb6b80 1 -- 172.21.15.6:0/3085209246 wait complete. 2023-11-17T00:10:40.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.157+0000 7f7156fb6b80 20 client.15858 trim_cache size 0 max 16384 2023-11-17T00:10:40.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 Processor -- start 2023-11-17T00:10:40.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 -- start start 2023-11-17T00:10:40.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd21a0f60 con 0x562fd21b4d40 2023-11-17T00:10:40.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd21b3890 con 0x562fd21afff0 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd219e9e0 con 0x562fd21b0900 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.175+0000 7f714139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59940/0 (socket says 172.21.15.6:59940) 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 -- 172.21.15.6:0/1732745096 learned_addr learned my addr 172.21.15.6:0/1732745096 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f7141b9c700 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f7140b9a700 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 -- 172.21.15.6:0/1732745096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 msgr2=0x562fd21aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 -- 172.21.15.6:0/1732745096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 msgr2=0x562fd21aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 -- 172.21.15.6:0/1732745096 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd21a82c0 con 0x562fd21afff0 2023-11-17T00:10:40.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f714139b700 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7138013010 tx=0x7f7138014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b222dbcd549075d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f7127fff700 1 -- 172.21.15.6:0/1732745096 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138022030 con 0x562fd21afff0 2023-11-17T00:10:40.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f7127fff700 1 -- 172.21.15.6:0/1732745096 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7138023040 con 0x562fd21afff0 2023-11-17T00:10:40.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.176+0000 7f7127fff700 1 -- 172.21.15.6:0/1732745096 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138003e80 con 0x562fd21afff0 2023-11-17T00:10:40.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 -- 172.21.15.6:0/1732745096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 msgr2=0x562fd21a6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7138013010 tx=0x7f7138014000 comp rx=0 tx=0).stop 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 -- 172.21.15.6:0/1732745096 shutdown_connections 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd21aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21afff0 0x562fd21a6d60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/1732745096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21b0900 0x562fd21aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 -- 172.21.15.6:0/1732745096 >> 172.21.15.6:0/1732745096 conn(0x562fd21a13e0 msgr2=0x562fd21b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 -- 172.21.15.6:0/1732745096 shutdown_connections 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.177+0000 7f7156fb6b80 1 -- 172.21.15.6:0/1732745096 wait complete. 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.178+0000 7f7156fb6b80 1 -- start start 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.178+0000 7f7156fb6b80 1 Processor -- start 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd22176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd2211a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd2212040 con 0x562fd21b4d40 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd22121b0 con 0x562fd21b0900 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd2212370 con 0x562fd21afff0 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7156fb6b80 10 client.0 _ll_register_callbacks cb 0x562fd20bc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7140b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7140b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59956/0 (socket says 172.21.15.6:59956) 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7140b9a700 1 -- 172.21.15.6:0/2075476954 learned_addr learned my addr 172.21.15.6:0/2075476954 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f714139b700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd22176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7141b9c700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd2211a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.179+0000 7f7140b9a700 1 -- 172.21.15.6:0/2075476954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 msgr2=0x562fd22176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7140b9a700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd22176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7140b9a700 1 -- 172.21.15.6:0/2075476954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 msgr2=0x562fd2211a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7140b9a700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd2211a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7140b9a700 1 -- 172.21.15.6:0/2075476954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7138017070 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7140b9a700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f712c09a760 tx=0x7f712c085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea549a7d078661a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7124ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c005190 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f713400f040 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f712c09a7d0 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f712c09e800 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7156fb6b80 10 client.25925 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.180+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562fd1f2ae60 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c0c8080 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f712c00fad0 con 0x562fd21b0900 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:40.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 20 client.25925 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:40.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd221f1a0 0x562fd22215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:40.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 4 client.25925 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:10:40.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562fd20cb6e0 con 0x562fd221f1a0 2023-11-17T00:10:40.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.181+0000 7f714139b700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd221f1a0 0x562fd22215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:40.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.182+0000 7f7124ff9700 10 client.25925 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:40.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.182+0000 7f714139b700 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd221f1a0 0x562fd22215b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a6234e3c1a26af server_cookie=dfe55be096542831 in_seq=0 out_seq=0 2023-11-17T00:10:40.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.183+0000 7f7124ff9700 1 -- 172.21.15.6:0/2075476954 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562fd20cb6e0 con 0x562fd221f1a0 2023-11-17T00:10:40.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.183+0000 7f7124ff9700 10 client.25925 handle_command_reply: tid=0 2023-11-17T00:10:40.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 client.25925 shutdown 2023-11-17T00:10:40.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 msgr2=0x562fd2217bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f712c09a760 tx=0x7f712c085040 comp rx=0 tx=0).stop 2023-11-17T00:10:40.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 shutdown_connections 2023-11-17T00:10:40.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562fd221f1a0 0x562fd22215b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562fd21b4d40 0x562fd2211a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562fd21afff0 0x562fd22176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 --2- 172.21.15.6:0/2075476954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562fd21b0900 0x562fd2217bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7124ff9700 0 client.25925 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:40.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 >> 172.21.15.6:0/2075476954 conn(0x562fd21a13e0 msgr2=0x562fd21b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 shutdown_connections 2023-11-17T00:10:40.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 1 -- 172.21.15.6:0/2075476954 wait complete. 2023-11-17T00:10:40.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.184+0000 7f7156fb6b80 20 client.25925 trim_cache size 0 max 16384 2023-11-17T00:10:40.203 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:40.204 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:10:40.204 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "897a6476-9cfa-43fe-9802-86df2d89063c", 2023-11-17T00:10:40.204 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:10:40.205 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:40.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 msgr2=0x7f71280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 0x7f71280a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7138009c60 tx=0x7f7138009260 comp rx=0 tx=0).stop 2023-11-17T00:10:40.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 msgr2=0x7f713c196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:40.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f712c009990 tx=0x7f712c00f690 comp rx=0 tx=0).stop 2023-11-17T00:10:40.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 shutdown_connections 2023-11-17T00:10:40.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f71280a1340 0x7f71280a37f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f713c105560 0x7f713c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713c100c50 0x7f713c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 --2- 172.21.15.6:0/659701445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c1015f0 0x7f713c196950 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:40.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 >> 172.21.15.6:0/659701445 conn(0x7f713c077500 msgr2=0x7f713c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:40.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 shutdown_connections 2023-11-17T00:10:40.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:40.185+0000 7f71435ff700 1 -- 172.21.15.6:0/659701445 wait complete. 2023-11-17T00:10:40.245 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:10:40.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:40.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:40.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:40.663 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:40.686 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:40.687 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:40.687 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 69 -lt 100 ']' 2023-11-17T00:10:40.687 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 69 + 1 2023-11-17T00:10:40.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=70 2023-11-17T00:10:40.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:40.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:40.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:40.743 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:40.744 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:40.744 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:41.167 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:41.168 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: pgmap v56: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:40 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:10:41.283 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:41.283 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:41.304 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:41.304 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:41.304 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 70 -lt 100 ']' 2023-11-17T00:10:41.304 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 70 + 1 2023-11-17T00:10:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: pgmap v56: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-17T00:10:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:10:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:10:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:40 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:10:41.357 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=71 2023-11-17T00:10:41.357 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: pgmap v56: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-17T00:10:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:10:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:10:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:41.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:40 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:10:41.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:41.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:41.400 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:41.400 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:41.400 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:42.176 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:42.176 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:42.310 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:42.310 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:42.321 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:42.321 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:42.322 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 71 -lt 100 ']' 2023-11-17T00:10:42.322 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 71 + 1 2023-11-17T00:10:42.357 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=72 2023-11-17T00:10:42.357 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:42.425 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:42.425 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:42.428 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:42.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:42.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:43.176 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:43.177 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:42 smithi006 ceph-mon[97840]: pgmap v57: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:42 smithi087 ceph-mon[131182]: pgmap v57: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:43.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:42 smithi186 ceph-mon[131487]: pgmap v57: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:43.393 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:43.393 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:43.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:43.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:43.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 72 -lt 100 ']' 2023-11-17T00:10:43.422 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 72 + 1 2023-11-17T00:10:43.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=73 2023-11-17T00:10:43.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:43.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:43.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:43.523 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:43.525 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:43.525 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 -- 172.21.15.6:0/2266393890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa0100c50 msgr2=0x7f7fa0101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/2266393890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa0100c50 0x7f7fa0101020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0099c0 tx=0x7f7f9c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 -- 172.21.15.6:0/2266393890 shutdown_connections 2023-11-17T00:10:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/2266393890 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa0100c50 0x7f7fa0101020 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/2266393890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0105560 0x7f7fa010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/2266393890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01015f0 0x7f7fa0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 -- 172.21.15.6:0/2266393890 >> 172.21.15.6:0/2266393890 conn(0x7f7fa0077500 msgr2=0x7f7fa0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 -- 172.21.15.6:0/2266393890 shutdown_connections 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.762+0000 7f7fa7fff700 1 -- 172.21.15.6:0/2266393890 wait complete. 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 Processor -- start 2023-11-17T00:10:43.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 -- start start 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100c50 0x7f7fa0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa0105560 0x7f7fa01a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0073b60 con 0x7f7fa01015f0 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4ab0 con 0x7f7fa0105560 2023-11-17T00:10:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa7fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4bf0 con 0x7f7fa0100c50 2023-11-17T00:10:43.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.764+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51582/0 (socket says 172.21.15.6:51582) 2023-11-17T00:10:43.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 -- 172.21.15.6:0/445578663 learned_addr learned my addr 172.21.15.6:0/445578663 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:43.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa659c700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa0105560 0x7f7fa01a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100c50 0x7f7fa0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 -- 172.21.15.6:0/445578663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100c50 msgr2=0x7f7fa0072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100c50 0x7f7fa0072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 -- 172.21.15.6:0/445578663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa0105560 msgr2=0x7f7fa01a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa0105560 0x7f7fa01a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 -- 172.21.15.6:0/445578663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009520 con 0x7f7fa01015f0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.765+0000 7f7fa559a700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009fd0 tx=0x7f7f9000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.766+0000 7f7f96ffd700 1 -- 172.21.15.6:0/445578663 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000cbd0 con 0x7f7fa01015f0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.766+0000 7f7f96ffd700 1 -- 172.21.15.6:0/445578663 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f9000eea0 con 0x7f7fa01015f0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.766+0000 7f7f96ffd700 1 -- 172.21.15.6:0/445578663 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003cf0 con 0x7f7fa01015f0 2023-11-17T00:10:43.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.766+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c0095f0 con 0x7f7fa01015f0 2023-11-17T00:10:43.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.766+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa0110360 con 0x7f7fa01015f0 2023-11-17T00:10:43.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.769+0000 7f7f96ffd700 1 -- 172.21.15.6:0/445578663 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7f90003eb0 con 0x7f7fa01015f0 2023-11-17T00:10:43.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.771+0000 7f7f96ffd700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 0x7f7f8c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.771+0000 7f7f96ffd700 1 -- 172.21.15.6:0/445578663 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7f90014070 con 0x7f7fa01015f0 2023-11-17T00:10:43.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.771+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 0x7f7f8c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.772+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 0x7f7f8c0a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009990 tx=0x7f7f9c005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 Processor -- start 2023-11-17T00:10:43.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 -- start start 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a927ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a92838b0 0x55b1a9283d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a9184060 con 0x55b1a92838b0 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a91d68a0 con 0x55b1a927eb60 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.859+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a91cfd30 con 0x55b1a927f470 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59438/0 (socket says 172.21.15.6:59438) 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 -- 172.21.15.6:0/3590681487 learned_addr learned my addr 172.21.15.6:0/3590681487 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a927ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa659c700 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a92838b0 0x55b1a9283d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 -- 172.21.15.6:0/3590681487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 msgr2=0x55b1a927ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a927ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 -- 172.21.15.6:0/3590681487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a92838b0 msgr2=0x55b1a9283d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a92838b0 0x55b1a9283d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.860+0000 7f7fa559a700 1 -- 172.21.15.6:0/3590681487 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1a9051630 con 0x55b1a927f470 2023-11-17T00:10:43.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.861+0000 7f7fa559a700 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f90096510 tx=0x7f7f900098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a818c28142baa76 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.861+0000 7f7f8bfff700 1 -- 172.21.15.6:0/3590681487 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900a2460 con 0x55b1a927f470 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.861+0000 7f7f8bfff700 1 -- 172.21.15.6:0/3590681487 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f9000cd30 con 0x55b1a927f470 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.861+0000 7f7f8bfff700 1 -- 172.21.15.6:0/3590681487 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90099500 con 0x55b1a927f470 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/3590681487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 msgr2=0x55b1a9283370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f90096510 tx=0x7f7f900098e0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/3590681487 shutdown_connections 2023-11-17T00:10:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a92838b0 0x55b1a9283d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a927ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/3590681487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9283370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/3590681487 >> 172.21.15.6:0/3590681487 conn(0x55b1a9274360 msgr2=0x55b1a9279b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/3590681487 shutdown_connections 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.862+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/3590681487 wait complete. 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.863+0000 7f7fbb9c6b80 1 -- start start 2023-11-17T00:10:43.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.863+0000 7f7fbb9c6b80 1 Processor -- start 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.863+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a928b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.863+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 0x55b1a92882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a9288940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a91718e0 con 0x55b1a927f470 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a9171580 con 0x55b1a927eb60 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a9053920 con 0x55b1a92838b0 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fbb9c6b80 10 client.0 _ll_register_callbacks cb 0x55b1a918f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 0x55b1a92882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a9288940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a9288940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59444/0 (socket says 172.21.15.6:59444) 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a928b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa659c700 1 -- 172.21.15.6:0/1278050754 learned_addr learned my addr 172.21.15.6:0/1278050754 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 -- 172.21.15.6:0/1278050754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 msgr2=0x55b1a9288940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a9288940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 -- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 msgr2=0x55b1a928b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a928b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.864+0000 7f7fa559a700 1 -- 172.21.15.6:0/1278050754 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa0110360 con 0x55b1a927f470 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7fa559a700 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 0x55b1a92882c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000f010 tx=0x7f7f90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfe9ae834e19761 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7f897fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900b8070 con 0x55b1a927f470 2023-11-17T00:10:43.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7f900a2460 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1a9271b20 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f90096350 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7fbb9c6b80 10 client.15876 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.865+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b1a9196190 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f9007c020 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f90004160 con 0x55b1a927f470 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 20 client.15876 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a9298460 0x55b1a929a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 4 client.15876 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1a91920a0 con 0x55b1a9298460 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.866+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a9298460 0x55b1a929a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.867+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a9298460 0x55b1a929a840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbdd72ba3fbbb95e server_cookie=7dc0bb21797a056b in_seq=0 out_seq=0 2023-11-17T00:10:43.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.867+0000 7f7f897fa700 10 client.15876 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:43.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7f897fa700 1 -- 172.21.15.6:0/1278050754 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b1a91920a0 con 0x55b1a9298460 2023-11-17T00:10:43.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7f897fa700 10 client.15876 handle_command_reply: tid=0 2023-11-17T00:10:43.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 client.15876 shutdown 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 msgr2=0x55b1a92882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 0x55b1a92882c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000f010 tx=0x7f7f90000bc0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 shutdown_connections 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a9288940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927eb60 0x55b1a928b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a9298460 0x55b1a929a840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/1278050754 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927f470 0x55b1a92882c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 >> 172.21.15.6:0/1278050754 conn(0x55b1a9274360 msgr2=0x55b1a9284d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7f897fa700 0 client.15876 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:43.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 shutdown_connections 2023-11-17T00:10:43.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.868+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/1278050754 wait complete. 2023-11-17T00:10:43.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.869+0000 7f7fbb9c6b80 20 client.15876 trim_cache size 0 max 16384 2023-11-17T00:10:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.888+0000 7f7fbb9c6b80 1 Processor -- start 2023-11-17T00:10:43.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 -- start start 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9294be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a92838b0 0x55b1a928abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a927fc10 con 0x55b1a927eb60 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a927b1f0 con 0x55b1a92838b0 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a9286820 con 0x55b1a927f470 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51614/0 (socket says 172.21.15.6:51614) 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.889+0000 7f7fa5d9b700 1 -- 172.21.15.6:0/610994447 learned_addr learned my addr 172.21.15.6:0/610994447 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa659c700 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a92838b0 0x55b1a928abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa559a700 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9294be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 -- 172.21.15.6:0/610994447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 msgr2=0x55b1a9294be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9294be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 -- 172.21.15.6:0/610994447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a92838b0 msgr2=0x55b1a928abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a92838b0 0x55b1a928abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 -- 172.21.15.6:0/610994447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1a9195ee0 con 0x55b1a927eb60 2023-11-17T00:10:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0036c0 tx=0x7f7f9c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d5a7204c4b60d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.890+0000 7f7f94ff9700 1 -- 172.21.15.6:0/610994447 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c02c040 con 0x55b1a927eb60 2023-11-17T00:10:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7f94ff9700 1 -- 172.21.15.6:0/610994447 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f9c02d040 con 0x55b1a927eb60 2023-11-17T00:10:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7f94ff9700 1 -- 172.21.15.6:0/610994447 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c004260 con 0x55b1a927eb60 2023-11-17T00:10:43.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/610994447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 msgr2=0x55b1a928f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0036c0 tx=0x7f7f9c022000 comp rx=0 tx=0).stop 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/610994447 shutdown_connections 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f9a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a92838b0 0x55b1a928abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/610994447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a927f470 0x55b1a9294be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/610994447 >> 172.21.15.6:0/610994447 conn(0x55b1a9274360 msgr2=0x55b1a9278550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/610994447 shutdown_connections 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.891+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/610994447 wait complete. 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.892+0000 7f7fbb9c6b80 1 -- start start 2023-11-17T00:10:43.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.892+0000 7f7fbb9c6b80 1 Processor -- start 2023-11-17T00:10:43.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a928c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a928c560 con 0x55b1a927eb60 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a928c6d0 con 0x55b1a927f470 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1a928c890 con 0x55b1a92838b0 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fbb9c6b80 10 client.0 _ll_register_callbacks cb 0x7f7f98009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fa5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51622/0 (socket says 172.21.15.6:51622) 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.893+0000 7f7fa559a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:60004/0 (socket says 172.21.15.6:60004) 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa5d9b700 1 -- 172.21.15.6:0/395917076 learned_addr learned my addr 172.21.15.6:0/395917076 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa659c700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a928c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 -- 172.21.15.6:0/395917076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 msgr2=0x55b1a928c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a928c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 -- 172.21.15.6:0/395917076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 msgr2=0x55b1a928f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 -- 172.21.15.6:0/395917076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa010d940 con 0x55b1a927f470 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7fa559a700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f900999e0 tx=0x7f7f900045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e887dd7ebda182b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.894+0000 7f7f897fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900b8040 con 0x55b1a927f470 2023-11-17T00:10:43.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7f9007d080 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f9000cd30 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f900a2460 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 10 client.25943 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b1a8fe6c60 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f9000cd30 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f90098f70 con 0x55b1a927f470 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 20 client.25943 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a92ed200 0x55b1a92ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 4 client.25943 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.895+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b1a919e660 con 0x55b1a92ed200 2023-11-17T00:10:43.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.896+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a92ed200 0x55b1a92ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.896+0000 7f7fa5d9b700 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a92ed200 0x55b1a92ef5e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d3eb342c9def05 server_cookie=7bdbd137c99be97b in_seq=0 out_seq=0 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.896+0000 7f7f897fa700 10 client.25943 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7f897fa700 1 -- 172.21.15.6:0/395917076 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b1a919e660 con 0x55b1a92ed200 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7f897fa700 10 client.25943 handle_command_reply: tid=0 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 client.25943 shutdown 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 msgr2=0x55b1a928bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f900999e0 tx=0x7f7f900045e0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 shutdown_connections 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b1a92ed200 0x55b1a92ef5e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b1a927eb60 0x55b1a928f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1a92838b0 0x55b1a928c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 --2- 172.21.15.6:0/395917076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b1a927f470 0x55b1a928bae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7f897fa700 0 client.25943 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 >> 172.21.15.6:0/395917076 conn(0x55b1a9274360 msgr2=0x55b1a9294be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 shutdown_connections 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 1 -- 172.21.15.6:0/395917076 wait complete. 2023-11-17T00:10:43.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.897+0000 7f7fbb9c6b80 20 client.25943 trim_cache size 0 max 16384 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 msgr2=0x7f7f8c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 0x7f7f8c0a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009990 tx=0x7f7f9c005fb0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 msgr2=0x7f7fa0073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009fd0 tx=0x7f7f9000ed10 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 shutdown_connections 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7f8c0a1340 0x7f7f8c0a37f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa0105560 0x7f7fa01a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100c50 0x7f7fa0072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 --2- 172.21.15.6:0/445578663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fa01015f0 0x7f7fa0073330 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 >> 172.21.15.6:0/445578663 conn(0x7f7fa0077500 msgr2=0x7f7fa00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 shutdown_connections 2023-11-17T00:10:43.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:43.898+0000 7f7fa7fff700 1 -- 172.21.15.6:0/445578663 wait complete. 2023-11-17T00:10:43.955 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:10:43.956 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:10:44.300 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:44.300 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:44.378 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:44.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:44.395 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:44.395 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:44.395 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 73 -lt 100 ']' 2023-11-17T00:10:44.396 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 73 + 1 2023-11-17T00:10:44.423 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=74 2023-11-17T00:10:44.424 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:44.440 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:44.440 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:44.443 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:44.443 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:44.443 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:44.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:44.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:44.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:44.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:44.886 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:44.887 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:44.887 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 74 -lt 100 ']' 2023-11-17T00:10:44.887 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 74 + 1 2023-11-17T00:10:44.916 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=75 2023-11-17T00:10:44.916 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:44.947 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:44.947 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:44.949 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:44.950 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:44.950 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:45.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:44 smithi006 ceph-mon[97840]: pgmap v58: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:44 smithi087 ceph-mon[131182]: pgmap v58: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:44 smithi186 ceph-mon[131487]: pgmap v58: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-11-17T00:10:45.600 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:45.600 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:45.836 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:45.836 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:45.844 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:45.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:45.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 75 -lt 100 ']' 2023-11-17T00:10:45.845 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 75 + 1 2023-11-17T00:10:45.898 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=76 2023-11-17T00:10:45.898 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:45.943 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:45.943 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:45.945 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:45.946 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:45.947 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:46.767 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:46.767 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.794+0000 7f173b3ff700 1 -- 172.21.15.6:0/40195906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 msgr2=0x7f1734101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.794+0000 7f173b3ff700 1 --2- 172.21.15.6:0/40195906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f1734101090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17300099c0 tx=0x7f1730009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 -- 172.21.15.6:0/40195906 shutdown_connections 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 --2- 172.21.15.6:0/40195906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f1734101090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 --2- 172.21.15.6:0/40195906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 0x7f173410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 --2- 172.21.15.6:0/40195906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 0x7f1734104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 -- 172.21.15.6:0/40195906 >> 172.21.15.6:0/40195906 conn(0x7f17340774d0 msgr2=0x7f17340778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 -- 172.21.15.6:0/40195906 shutdown_connections 2023-11-17T00:10:46.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.795+0000 7f173b3ff700 1 -- 172.21.15.6:0/40195906 wait complete. 2023-11-17T00:10:46.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 Processor -- start 2023-11-17T00:10:46.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 -- start start 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 0x7f1734198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 0x7f173419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173419c5c0 con 0x7f1734100cc0 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173419c730 con 0x7f1734101660 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.796+0000 7f173b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173419c8f0 con 0x7f1734105620 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173899a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 0x7f1734198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51640/0 (socket says 172.21.15.6:51640) 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 -- 172.21.15.6:0/1559126082 learned_addr learned my addr 172.21.15.6:0/1559126082 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173999c700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 0x7f173419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:46.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 -- 172.21.15.6:0/1559126082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 msgr2=0x7f173419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 0x7f173419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 -- 172.21.15.6:0/1559126082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 msgr2=0x7f1734198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 0x7f1734198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 -- 172.21.15.6:0/1559126082 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724009520 con 0x7f1734100cc0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.797+0000 7f173919b700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f173000f690 tx=0x7f173000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.798+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173001c070 con 0x7f1734100cc0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.798+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f173000fd70 con 0x7f1734100cc0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.798+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730022c40 con 0x7f1734100cc0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.798+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17300095f0 con 0x7f1734100cc0 2023-11-17T00:10:46.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.798+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17341103d0 con 0x7f1734100cc0 2023-11-17T00:10:46.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.799+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1734110240 con 0x7f1734100cc0 2023-11-17T00:10:46.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.801+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1730020070 con 0x7f1734100cc0 2023-11-17T00:10:46.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.802+0000 7f172a7fc700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 0x7f17200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:46.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.802+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f17300cdf10 con 0x7f1734100cc0 2023-11-17T00:10:46.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.803+0000 7f173899a700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 0x7f17200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:46.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.803+0000 7f173899a700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 0x7f17200a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1734199ce0 tx=0x7f1724009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:46.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:46.818+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1730097000 con 0x7f1734100cc0 2023-11-17T00:10:47.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.076+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1734110560 con 0x7f1734100cc0 2023-11-17T00:10:47.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.077+0000 7f172a7fc700 1 -- 172.21.15.6:0/1559126082 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f173009a620 con 0x7f1734100cc0 2023-11-17T00:10:47.079 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:10:47.079 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:10:47.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.079+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 msgr2=0x7f17200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.079+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 0x7f17200a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1734199ce0 tx=0x7f1724009260 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 msgr2=0x7f17341986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f173000f690 tx=0x7f173000bfd0 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 shutdown_connections 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f17200a1340 0x7f17200a37f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1734100cc0 0x7f17341986d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1734105620 0x7f173419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 --2- 172.21.15.6:0/1559126082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734101660 0x7f1734198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 >> 172.21.15.6:0/1559126082 conn(0x7f17340774d0 msgr2=0x7f17340feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 shutdown_connections 2023-11-17T00:10:47.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:47.080+0000 7f173b3ff700 1 -- 172.21.15.6:0/1559126082 wait complete. 2023-11-17T00:10:47.083 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:10:47.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:10:47.170 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:47.170 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:47.187 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:47.187 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:47.187 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 76 -lt 100 ']' 2023-11-17T00:10:47.188 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 76 + 1 2023-11-17T00:10:47.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:46 smithi006 ceph-mon[97840]: pgmap v59: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-11-17T00:10:47.223 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=77 2023-11-17T00:10:47.223 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:47.240 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:47.240 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:47.242 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:47.242 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:47.243 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:47.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:46 smithi087 ceph-mon[131182]: pgmap v59: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-11-17T00:10:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:46 smithi186 ceph-mon[131487]: pgmap v59: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-11-17T00:10:47.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:47.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:47.784 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:47.785 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:47.794 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:47.795 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:47.795 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 77 -lt 100 ']' 2023-11-17T00:10:47.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 77 + 1 2023-11-17T00:10:47.849 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=78 2023-11-17T00:10:47.849 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:47.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:47.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:47.911 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:47.912 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:47.912 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:48.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:47 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1559126082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:48.325 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:48.326 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:48.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:47 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1559126082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:48.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:47 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1559126082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:10:48.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:48.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:48.553 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:48.553 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:48.554 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 78 -lt 100 ']' 2023-11-17T00:10:48.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 78 + 1 2023-11-17T00:10:48.598 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=79 2023-11-17T00:10:48.598 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:48.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:48.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:48.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:48.648 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:48.649 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:48 smithi006 ceph-mon[97840]: pgmap v60: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-11-17T00:10:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:48 smithi087 ceph-mon[131182]: pgmap v60: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-11-17T00:10:49.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:48 smithi186 ceph-mon[131487]: pgmap v60: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-11-17T00:10:49.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:49.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:49.526 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:49.526 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:49.571 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:49.571 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:49.571 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 79 -lt 100 ']' 2023-11-17T00:10:49.571 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 79 + 1 2023-11-17T00:10:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=80 2023-11-17T00:10:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:49.661 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:49.662 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:49.665 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:49.665 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:49.666 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 -- 172.21.15.6:0/67655810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8105250 msgr2=0x7f2ef8105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 --2- 172.21.15.6:0/67655810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8105250 0x7f2ef8105620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80099c0 tx=0x7f2ee8009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 -- 172.21.15.6:0/67655810 shutdown_connections 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 --2- 172.21.15.6:0/67655810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8105250 0x7f2ef8105620 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 --2- 172.21.15.6:0/67655810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 0x7f2ef810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 --2- 172.21.15.6:0/67655810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8067dc0 0x7f2ef8068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 -- 172.21.15.6:0/67655810 >> 172.21.15.6:0/67655810 conn(0x7f2ef80763a0 msgr2=0x7f2ef80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 -- 172.21.15.6:0/67655810 shutdown_connections 2023-11-17T00:10:50.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.003+0000 7f2efcfff700 1 -- 172.21.15.6:0/67655810 wait complete. 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.004+0000 7f2efcfff700 1 Processor -- start 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.004+0000 7f2efcfff700 1 -- start start 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 0x7f2ef8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8105250 0x7f2ef819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d5d0 con 0x7f2ef8067dc0 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d740 con 0x7f2ef8105250 2023-11-17T00:10:50.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2efcfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d900 con 0x7f2ef8068770 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58776/0 (socket says 172.21.15.6:58776) 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 -- 172.21.15.6:0/2915876125 learned_addr learned my addr 172.21.15.6:0/2915876125 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 0x7f2ef8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8105250 0x7f2ef819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 -- 172.21.15.6:0/2915876125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 msgr2=0x7f2ef8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 0x7f2ef8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 -- 172.21.15.6:0/2915876125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8105250 msgr2=0x7f2ef819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8105250 0x7f2ef819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.005+0000 7f2ef659c700 1 -- 172.21.15.6:0/2915876125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80095f0 con 0x7f2ef8067dc0 2023-11-17T00:10:50.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2ee77fe700 1 -- 172.21.15.6:0/2915876125 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee801d070 con 0x7f2ef8067dc0 2023-11-17T00:10:50.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2ee77fe700 1 -- 172.21.15.6:0/2915876125 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee8005030 con 0x7f2ef8067dc0 2023-11-17T00:10:50.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef8110310 con 0x7f2ef8067dc0 2023-11-17T00:10:50.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2ee77fe700 1 -- 172.21.15.6:0/2915876125 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004a50 con 0x7f2ef8067dc0 2023-11-17T00:10:50.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.006+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef8110180 con 0x7f2ef8067dc0 2023-11-17T00:10:50.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.009+0000 7f2ee77fe700 1 -- 172.21.15.6:0/2915876125 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2ee8005430 con 0x7f2ef8067dc0 2023-11-17T00:10:50.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.011+0000 7f2ee77fe700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 0x7f2ee00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.011+0000 7f2ee77fe700 1 -- 172.21.15.6:0/2915876125 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2ee80cdd70 con 0x7f2ef8067dc0 2023-11-17T00:10:50.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.011+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 0x7f2ee00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.012+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 0x7f2ee00a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8199c00 tx=0x7f2eec009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.138+0000 7f2f109ecb80 1 Processor -- start 2023-11-17T00:10:50.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.138+0000 7f2f109ecb80 1 -- start start 2023-11-17T00:10:50.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad60370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad608b0 0x557eaad60d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaac61060 con 0x557eaad5bb60 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaacb38a0 con 0x557eaad608b0 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaacacd30 con 0x557eaad5c470 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58784/0 (socket says 172.21.15.6:58784) 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 -- 172.21.15.6:0/3818330338 learned_addr learned my addr 172.21.15.6:0/3818330338 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad60370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad608b0 0x557eaad60d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 -- 172.21.15.6:0/3818330338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 msgr2=0x557eaad60370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad60370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 -- 172.21.15.6:0/3818330338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad608b0 msgr2=0x557eaad60d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad608b0 0x557eaad60d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.139+0000 7f2ef659c700 1 -- 172.21.15.6:0/3818330338 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eaab2e630 con 0x557eaad5bb60 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.140+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ee809aab0 tx=0x7f2ee809aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcb5d5cad91bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.140+0000 7f2ee4ff9700 1 -- 172.21.15.6:0/3818330338 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80a2ee0 con 0x557eaad5bb60 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.140+0000 7f2ee4ff9700 1 -- 172.21.15.6:0/3818330338 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee8098ec0 con 0x557eaad5bb60 2023-11-17T00:10:50.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.140+0000 7f2ee4ff9700 1 -- 172.21.15.6:0/3818330338 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee809a230 con 0x557eaad5bb60 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3818330338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 msgr2=0x557eaad5bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ee809aab0 tx=0x7f2ee809aba0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3818330338 shutdown_connections 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5bb60 0x557eaad5bf30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad608b0 0x557eaad60d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3818330338 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad60370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3818330338 >> 172.21.15.6:0/3818330338 conn(0x557eaad51360 msgr2=0x557eaad56b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3818330338 shutdown_connections 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3818330338 wait complete. 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.141+0000 7f2f109ecb80 1 -- start start 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 Processor -- start 2023-11-17T00:10:50.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad68010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad608b0 0x557eaad65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaac4e8e0 con 0x557eaad5c470 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaac4e580 con 0x557eaad5bb60 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaab30920 con 0x557eaad608b0 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.142+0000 7f2f109ecb80 10 client.0 _ll_register_callbacks cb 0x557eaac6c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad68010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58796/0 (socket says 172.21.15.6:58796) 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 -- 172.21.15.6:0/2152471977 learned_addr learned my addr 172.21.15.6:0/2152471977 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad608b0 0x557eaad65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 -- 172.21.15.6:0/2152471977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad608b0 msgr2=0x557eaad65830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad608b0 0x557eaad65830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 -- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 msgr2=0x557eaad68010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad68010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 -- 172.21.15.6:0/2152471977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef8110180 con 0x557eaad5c470 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad68010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0045d0 tx=0x7f2eec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f18808906f5cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.143+0000 7f2eddffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec022070 con 0x557eaad5c470 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2eec003e20 con 0x557eaad5c470 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557eaad4eb20 con 0x557eaad5c470 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eec01d620 con 0x557eaad5c470 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2f109ecb80 10 client.15900 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:50.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557eaac73190 con 0x557eaad5c470 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557eaad4eb20 con 0x557eaad5c470 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2eec027780 con 0x557eaad5c470 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2f109ecb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.144+0000 7f2f109ecb80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.145+0000 7f2f109ecb80 20 client.15900 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.145+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaad75620 0x557eaad77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.145+0000 7f2f109ecb80 4 client.15900 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.145+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557eaac6f0a0 con 0x557eaad75620 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.145+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaad75620 0x557eaad77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.146+0000 7f2ef659c700 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaad75620 0x557eaad77a30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e80e01f43f24f24 server_cookie=c4fc108ded5b5e2 in_seq=0 out_seq=0 2023-11-17T00:10:50.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.146+0000 7f2eddffb700 10 client.15900 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:50.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.146+0000 7f2eddffb700 1 -- 172.21.15.6:0/2152471977 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557eaac6f0a0 con 0x557eaad75620 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.146+0000 7f2eddffb700 10 client.15900 handle_command_reply: tid=0 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 client.15900 shutdown 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 msgr2=0x557eaad65220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0045d0 tx=0x7f2eec0047c0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 shutdown_connections 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad608b0 0x557eaad65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad68010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaad75620 0x557eaad77a30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/2152471977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad5c470 0x557eaad65220 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2eddffb700 0 client.15900 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:50.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 >> 172.21.15.6:0/2152471977 conn(0x557eaad51360 msgr2=0x557eaad61d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 shutdown_connections 2023-11-17T00:10:50.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 1 -- 172.21.15.6:0/2152471977 wait complete. 2023-11-17T00:10:50.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.147+0000 7f2f109ecb80 20 client.15900 trim_cache size 0 max 16384 2023-11-17T00:10:50.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 Processor -- start 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 -- start start 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad6c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaac6f0a0 con 0x557eaad608b0 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaad584a0 con 0x557eaad5bb60 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaad72cd0 con 0x557eaad5c470 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2ef6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2ef6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58812/0 (socket says 172.21.15.6:58812) 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2ef6d9d700 1 -- 172.21.15.6:0/875374247 learned_addr learned my addr 172.21.15.6:0/875374247 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2ef659c700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.162+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad6c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 -- 172.21.15.6:0/875374247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 msgr2=0x557eaad6c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad6c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 -- 172.21.15.6:0/875374247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 msgr2=0x557eaad56c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad56c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 -- 172.21.15.6:0/875374247 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eaac72ee0 con 0x557eaad608b0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef659c700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad56c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009ed0 tx=0x7f2ef0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbceee61adcfdc2a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ee57fa700 1 -- 172.21.15.6:0/875374247 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0007e40 con 0x557eaad608b0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ee57fa700 1 -- 172.21.15.6:0/875374247 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef000f040 con 0x557eaad608b0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.163+0000 7f2ee57fa700 1 -- 172.21.15.6:0/875374247 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017410 con 0x557eaad608b0 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 -- 172.21.15.6:0/875374247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 msgr2=0x557eaad64ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009ed0 tx=0x7f2ef0009f00 comp rx=0 tx=0).stop 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 -- 172.21.15.6:0/875374247 shutdown_connections 2023-11-17T00:10:50.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad64ac0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5bb60 0x557eaad56c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/875374247 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5c470 0x557eaad6c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 -- 172.21.15.6:0/875374247 >> 172.21.15.6:0/875374247 conn(0x557eaad51360 msgr2=0x557eaad5e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 -- 172.21.15.6:0/875374247 shutdown_connections 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.164+0000 7f2f109ecb80 1 -- 172.21.15.6:0/875374247 wait complete. 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 -- start start 2023-11-17T00:10:50.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 Processor -- start 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5c470 0x557eaad639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaad654c0 con 0x557eaad608b0 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaad65630 con 0x557eaad5c470 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eaad64470 con 0x557eaad5bb60 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.165+0000 7f2f109ecb80 10 client.0 _ll_register_callbacks cb 0x7f2eec0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56592/0 (socket says 172.21.15.6:56592) 2023-11-17T00:10:50.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 -- 172.21.15.6:0/3180515045 learned_addr learned my addr 172.21.15.6:0/3180515045 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef6d9d700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5c470 0x557eaad639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 -- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5c470 msgr2=0x557eaad639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5c470 0x557eaad639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 -- 172.21.15.6:0/3180515045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 msgr2=0x557eaad63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad63f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 -- 172.21.15.6:0/3180515045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557eaab2e630 con 0x557eaad5bb60 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2ef659c700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8097020 tx=0x7f2ee80056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25df7168c99febda server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.166+0000 7f2eddffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80b3250 con 0x557eaad5bb60 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2ee8005220 con 0x557eaad5bb60 2023-11-17T00:10:50.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ee8098ec0 con 0x557eaad5bb60 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ee809ad80 con 0x557eaad5bb60 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 10 client.25948 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2ef0009b50 con 0x557eaad5bb60 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee807d020 con 0x557eaad5bb60 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ee8004bb0 con 0x557eaad5bb60 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 10 client.25948 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 20 client.25948 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaadca1a0 0x557eaadcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 4 client.25948 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:10:50.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.167+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557eaac7b660 con 0x557eaadca1a0 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaadca1a0 0x557eaadcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2ef5d9b700 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaadca1a0 0x557eaadcc580 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe1957426b03388 server_cookie=9c64fe75e36a5c13 in_seq=0 out_seq=0 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2eddffb700 10 client.25948 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2eddffb700 1 -- 172.21.15.6:0/3180515045 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557eaac7b660 con 0x557eaadca1a0 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2eddffb700 10 client.25948 handle_command_reply: tid=0 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2f109ecb80 1 client.25948 shutdown 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 msgr2=0x557eaad64f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8097020 tx=0x7f2ee80056e0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.168+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 shutdown_connections 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557eaadca1a0 0x557eaadcc580 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557eaad608b0 0x557eaad63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557eaad5bb60 0x557eaad64f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 --2- 172.21.15.6:0/3180515045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557eaad5c470 0x557eaad639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2eddffb700 0 client.25948 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:50.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 >> 172.21.15.6:0/3180515045 conn(0x557eaad51360 msgr2=0x557eaad62570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 shutdown_connections 2023-11-17T00:10:50.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 1 -- 172.21.15.6:0/3180515045 wait complete. 2023-11-17T00:10:50.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2f109ecb80 20 client.25948 trim_cache size 0 max 16384 2023-11-17T00:10:50.226 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 msgr2=0x7f2ee00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 0x7f2ee00a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8199c00 tx=0x7f2eec009260 comp rx=0 tx=0).stop 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 msgr2=0x7f2ef81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.169+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee800f690 comp rx=0 tx=0).stop 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 shutdown_connections 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f2ee00a1340 0x7f2ee00a37f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ef8067dc0 0x7f2ef81985f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ef8105250 0x7f2ef819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 --2- 172.21.15.6:0/2915876125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ef8068770 0x7f2ef8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:50.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 >> 172.21.15.6:0/2915876125 conn(0x7f2ef80763a0 msgr2=0x7f2ef80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:50.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 shutdown_connections 2023-11-17T00:10:50.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:50.170+0000 7f2efcfff700 1 -- 172.21.15.6:0/2915876125 wait complete. 2023-11-17T00:10:50.228 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:49 smithi006 ceph-mon[97840]: pgmap v61: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-17T00:10:50.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:49 smithi087 ceph-mon[131182]: pgmap v61: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-17T00:10:50.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:49 smithi186 ceph-mon[131487]: pgmap v61: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-11-17T00:10:50.449 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:50.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:50.592 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:50.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:50.603 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:50.603 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:50.603 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 80 -lt 100 ']' 2023-11-17T00:10:50.604 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 80 + 1 2023-11-17T00:10:50.656 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=81 2023-11-17T00:10:50.656 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:50.686 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:50.687 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:50.693 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:50.693 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:50.693 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:51.237 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:10:51.237 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:10:51.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:51.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:51.737 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:51.737 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:51.762 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:51.762 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:51.762 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 81 -lt 100 ']' 2023-11-17T00:10:51.763 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 81 + 1 2023-11-17T00:10:51.814 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=82 2023-11-17T00:10:51.815 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:51.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:51.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:51.841 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:51.842 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:51.842 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.160+0000 7fbf783ff700 1 -- 172.21.15.6:0/2476700563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100540 msgr2=0x7fbf70100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.160+0000 7fbf783ff700 1 --2- 172.21.15.6:0/2476700563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100540 0x7fbf70100910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b0d0 tx=0x7fbf6c00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 -- 172.21.15.6:0/2476700563 shutdown_connections 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 --2- 172.21.15.6:0/2476700563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70105530 0x7fbf7010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 --2- 172.21.15.6:0/2476700563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100540 0x7fbf70100910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 --2- 172.21.15.6:0/2476700563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70100ee0 0x7fbf70104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 -- 172.21.15.6:0/2476700563 >> 172.21.15.6:0/2476700563 conn(0x7fbf700781e0 msgr2=0x7fbf700785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 -- 172.21.15.6:0/2476700563 shutdown_connections 2023-11-17T00:10:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.161+0000 7fbf783ff700 1 -- 172.21.15.6:0/2476700563 wait complete. 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 Processor -- start 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 -- start start 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 0x7fbf70072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 0x7fbf701a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70073b60 con 0x7fbf70105530 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701a4ca0 con 0x7fbf70100540 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.162+0000 7fbf783ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701a4de0 con 0x7fbf70100ee0 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7699c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 0x7fbf701a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7619b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 0x7fbf70072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56600/0 (socket says 172.21.15.6:56600) 2023-11-17T00:10:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 -- 172.21.15.6:0/3906211187 learned_addr learned my addr 172.21.15.6:0/3906211187 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7619b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 0x7fbf70072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40788/0 (socket says 172.21.15.6:40788) 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7699c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 0x7fbf701a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58846/0 (socket says 172.21.15.6:58846) 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 -- 172.21.15.6:0/3906211187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 msgr2=0x7fbf70072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 0x7fbf70072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 -- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 msgr2=0x7fbf701a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 0x7fbf701a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 -- 172.21.15.6:0/3906211187 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c009d00 con 0x7fbf70100ee0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.163+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e940 tx=0x7fbf6000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.164+0000 7fbf677fe700 1 -- 172.21.15.6:0/3906211187 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60009730 con 0x7fbf70100ee0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.164+0000 7fbf677fe700 1 -- 172.21.15.6:0/3906211187 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf60010430 con 0x7fbf70100ee0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.164+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf70110340 con 0x7fbf70100ee0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.164+0000 7fbf677fe700 1 -- 172.21.15.6:0/3906211187 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000cea0 con 0x7fbf70100ee0 2023-11-17T00:10:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.164+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf701101b0 con 0x7fbf70100ee0 2023-11-17T00:10:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.167+0000 7fbf677fe700 1 -- 172.21.15.6:0/3906211187 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbf60010810 con 0x7fbf70100ee0 2023-11-17T00:10:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.168+0000 7fbf677fe700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 0x7fbf5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.169+0000 7fbf677fe700 1 -- 172.21.15.6:0/3906211187 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf60014070 con 0x7fbf70100ee0 2023-11-17T00:10:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.169+0000 7fbf7619b700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 0x7fbf5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.169+0000 7fbf7619b700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 0x7fbf5c0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009fd0 tx=0x7fbf6c00b040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 Processor -- start 2023-11-17T00:10:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 -- start start 2023-11-17T00:10:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6b90 0x56284d4d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d3d6000 con 0x56284d4d6b90 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d428840 con 0x56284d32c6b0 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.280+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d421cd0 con 0x56284d4d6720 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7619b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56602/0 (socket says 172.21.15.6:56602) 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 -- 172.21.15.6:0/2306877751 learned_addr learned my addr 172.21.15.6:0/2306877751 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7699c700 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6b90 0x56284d4d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 -- 172.21.15.6:0/2306877751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 msgr2=0x56284d4d61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 -- 172.21.15.6:0/2306877751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6b90 msgr2=0x56284d4d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6b90 0x56284d4d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 -- 172.21.15.6:0/2306877751 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284d2a35d0 con 0x56284d4d6720 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf600969c0 tx=0x7fbf600999a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a34e4626457d70 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.281+0000 7fbf64bfe700 1 -- 172.21.15.6:0/2306877751 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600a23a0 con 0x56284d4d6720 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf64bfe700 1 -- 172.21.15.6:0/2306877751 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf600982f0 con 0x56284d4d6720 2023-11-17T00:10:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf64bfe700 1 -- 172.21.15.6:0/2306877751 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600a1f10 con 0x56284d4d6720 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2306877751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 msgr2=0x56284d4d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf600969c0 tx=0x7fbf600999a0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2306877751 shutdown_connections 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6b90 0x56284d4d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2306877751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6720 0x56284d4d0560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.282+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2306877751 >> 172.21.15.6:0/2306877751 conn(0x56284d4c5f50 msgr2=0x56284d4cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.283+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2306877751 shutdown_connections 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.283+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2306877751 wait complete. 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.283+0000 7fbf8bd6eb80 1 -- start start 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.283+0000 7fbf8bd6eb80 1 Processor -- start 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d4d6720 0x56284d4e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d3c3880 con 0x56284d32c6b0 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d3c3520 con 0x56284d4d6720 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d2a58c0 con 0x56284d4d6b90 2023-11-17T00:10:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf8bd6eb80 10 client.0 _ll_register_callbacks cb 0x56284d3e1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7619b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7619b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58862/0 (socket says 172.21.15.6:58862) 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7619b700 1 -- 172.21.15.6:0/2618262496 learned_addr learned my addr 172.21.15.6:0/2618262496 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7699c700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d4d6720 0x56284d4e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.284+0000 7fbf7619b700 1 -- 172.21.15.6:0/2618262496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 msgr2=0x56284d4d9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7619b700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7619b700 1 -- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d4d6720 msgr2=0x56284d4e0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7619b700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d4d6720 0x56284d4e0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7619b700 1 -- 172.21.15.6:0/2618262496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf701101b0 con 0x56284d32c6b0 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7699c700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf7619b700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007830 tx=0x7fbf6c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3288620265f0e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00e040 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf6c01c8a0 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56284d4c3710 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf6c01ca60 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf8bd6eb80 10 client.15918 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.285+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56284d3e8130 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56284d4c3710 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf6c029bb0 con 0x56284d32c6b0 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 20 client.15918 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d4ea010 0x56284d4ec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 4 client.15918 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.286+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56284d3e4040 con 0x56284d4ea010 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.287+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d4ea010 0x56284d4ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.287+0000 7fbf59ffb700 10 client.15918 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.287+0000 7fbf7599a700 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d4ea010 0x56284d4ec420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3529d7fbca5e45 server_cookie=1b9c016d4fd3d020 in_seq=0 out_seq=0 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf59ffb700 1 -- 172.21.15.6:0/2618262496 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56284d3e4040 con 0x56284d4ea010 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf59ffb700 10 client.15918 handle_command_reply: tid=0 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 client.15918 shutdown 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 msgr2=0x56284d4dfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007830 tx=0x7fbf6c0099e0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 shutdown_connections 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d4ea010 0x56284d4ec420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d32c6b0 0x56284d4dfb10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf59ffb700 0 client.15918 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/2618262496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d4d6720 0x56284d4e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 >> 172.21.15.6:0/2618262496 conn(0x56284d4c5f50 msgr2=0x56284d4ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 shutdown_connections 2023-11-17T00:10:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.288+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/2618262496 wait complete. 2023-11-17T00:10:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.289+0000 7fbf8bd6eb80 20 client.15918 trim_cache size 0 max 16384 2023-11-17T00:10:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 Processor -- start 2023-11-17T00:10:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 -- start start 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4c5b70 con 0x56284d4d6720 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4d82c0 con 0x56284d32c6b0 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.304+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4d4a10 con 0x56284d4d6b90 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7699c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7699c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56616/0 (socket says 172.21.15.6:56616) 2023-11-17T00:10:52.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7699c700 1 -- 172.21.15.6:0/3895509645 learned_addr learned my addr 172.21.15.6:0/3895509645 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7619b700 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 -- 172.21.15.6:0/3895509645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 msgr2=0x56284d4d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 -- 172.21.15.6:0/3895509645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 msgr2=0x56284d4d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7599a700 1 -- 172.21.15.6:0/3895509645 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284d3e7e80 con 0x56284d4d6720 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.305+0000 7fbf7699c700 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf7599a700 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4caf60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x56284d4d12a0 tx=0x7fbf6009e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68b1df7b48ffe3f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf653ff700 1 -- 172.21.15.6:0/3895509645 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600b81c0 con 0x56284d4d6720 2023-11-17T00:10:52.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf653ff700 1 -- 172.21.15.6:0/3895509645 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf600b8320 con 0x56284d4d6720 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf653ff700 1 -- 172.21.15.6:0/3895509645 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600a1f10 con 0x56284d4d6720 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/3895509645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 msgr2=0x56284d4caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4caf60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x56284d4d12a0 tx=0x7fbf6009e100 comp rx=0 tx=0).stop 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/3895509645 shutdown_connections 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/3895509645 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4caf60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.306+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/3895509645 >> 172.21.15.6:0/3895509645 conn(0x56284d4c5f50 msgr2=0x56284d4da690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.307+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/3895509645 shutdown_connections 2023-11-17T00:10:52.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.307+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/3895509645 wait complete. 2023-11-17T00:10:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.307+0000 7fbf8bd6eb80 1 -- start start 2023-11-17T00:10:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.307+0000 7fbf8bd6eb80 1 Processor -- start 2023-11-17T00:10:52.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.319 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4df310 con 0x56284d4d6720 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4df480 con 0x56284d32c6b0 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284d4df640 con 0x56284d4d6b90 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf8bd6eb80 10 client.0 _ll_register_callbacks cb 0x56284d3e1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf7599a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58870/0 (socket says 172.21.15.6:58870) 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf7599a700 1 -- 172.21.15.6:0/193283569 learned_addr learned my addr 172.21.15.6:0/193283569 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf7699c700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.308+0000 7fbf7619b700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 -- 172.21.15.6:0/193283569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 msgr2=0x56284d4ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.320 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4ded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 -- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 msgr2=0x56284d4dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 -- 172.21.15.6:0/193283569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf68012450 con 0x56284d4d6720 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7699c700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7619b700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf7599a700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e910 tx=0x7fbf60096390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4734385acdefe94b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6009a3b0 con 0x56284d4d6720 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf60099440 con 0x56284d4d6720 2023-11-17T00:10:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf600982f0 con 0x56284d4d6720 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf60099600 con 0x56284d4d6720 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf8bd6eb80 10 client.15924 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56284d2a35d0 con 0x56284d4d6720 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.309+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf600982f0 con 0x56284d4d6720 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf600cad20 con 0x56284d4d6720 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 20 client.15924 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d53ded0 0x56284d5402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 4 client.15924 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:10:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56284d3f0600 con 0x56284d53ded0 2023-11-17T00:10:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.310+0000 7fbf7619b700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d53ded0 0x56284d5402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.311+0000 7fbf7619b700 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d53ded0 0x56284d5402b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7eacd3b70e82341 server_cookie=d2b1bd9c5b6b4d59 in_seq=0 out_seq=0 2023-11-17T00:10:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.311+0000 7fbf59ffb700 10 client.15924 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.338+0000 7fbf59ffb700 1 -- 172.21.15.6:0/193283569 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56284d3f0600 con 0x56284d53ded0 2023-11-17T00:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.338+0000 7fbf59ffb700 10 client.15924 handle_command_reply: tid=0 2023-11-17T00:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.338+0000 7fbf8bd6eb80 1 client.15924 shutdown 2023-11-17T00:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 msgr2=0x56284d4de850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e910 tx=0x7fbf60096390 comp rx=0 tx=0).stop 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 shutdown_connections 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56284d4d6b90 0x56284d4ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56284d32c6b0 0x56284d4dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56284d53ded0 0x56284d5402b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 --2- 172.21.15.6:0/193283569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56284d4d6720 0x56284d4de850 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf59ffb700 0 client.15924 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 >> 172.21.15.6:0/193283569 conn(0x56284d4c5f50 msgr2=0x56284d4d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 shutdown_connections 2023-11-17T00:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 1 -- 172.21.15.6:0/193283569 wait complete. 2023-11-17T00:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.339+0000 7fbf8bd6eb80 20 client.15924 trim_cache size 0 max 16384 2023-11-17T00:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 msgr2=0x7fbf5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 0x7fbf5c0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009fd0 tx=0x7fbf6c00b040 comp rx=0 tx=0).stop 2023-11-17T00:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 msgr2=0x7fbf70073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e940 tx=0x7fbf6000ec50 comp rx=0 tx=0).stop 2023-11-17T00:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 shutdown_connections 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbf5c0a12c0 0x7fbf5c0a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf70105530 0x7fbf701a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf70100540 0x7fbf70072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 --2- 172.21.15.6:0/3906211187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf70100ee0 0x7fbf70073330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 >> 172.21.15.6:0/3906211187 conn(0x7fbf700781e0 msgr2=0x7fbf70103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 shutdown_connections 2023-11-17T00:10:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:10:52.340+0000 7fbf783ff700 1 -- 172.21.15.6:0/3906211187 wait complete. 2023-11-17T00:10:52.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:52 smithi006 ceph-mon[97840]: pgmap v62: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:52.800 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:52.800 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:52.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:52 smithi087 ceph-mon[131182]: pgmap v62: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:52.861 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:52.861 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:52.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:52 smithi186 ceph-mon[131487]: pgmap v62: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:52.894 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:52.894 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:52.894 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 82 -lt 100 ']' 2023-11-17T00:10:52.894 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 82 + 1 2023-11-17T00:10:52.942 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=83 2023-11-17T00:10:52.942 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:52.986 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:52.986 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:52.988 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:52.989 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:52.989 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:53.549 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:53.550 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:53.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:53.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:53.777 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:53.777 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:53.777 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 83 -lt 100 ']' 2023-11-17T00:10:53.778 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 83 + 1 2023-11-17T00:10:53.823 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=84 2023-11-17T00:10:53.823 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:53.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:53.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:53.881 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:53.882 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:53.882 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:54.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:54.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:54.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:54 smithi006 ceph-mon[97840]: pgmap v63: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:54 smithi087 ceph-mon[131182]: pgmap v63: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:54.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:54 smithi186 ceph-mon[131487]: pgmap v63: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-17T00:10:54.903 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:54.903 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:54.937 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:54.937 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:54.937 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 84 -lt 100 ']' 2023-11-17T00:10:54.937 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 84 + 1 2023-11-17T00:10:55.014 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=85 2023-11-17T00:10:55.014 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:55.158 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:55.159 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:55.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:55.162 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:55.162 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:55.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 -- 172.21.15.6:0/2010323368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4105560 msgr2=0x7f79a410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 --2- 172.21.15.6:0/2010323368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4105560 0x7f79a410d130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b0d0 tx=0x7f799c00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 -- 172.21.15.6:0/2010323368 shutdown_connections 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 --2- 172.21.15.6:0/2010323368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4105560 0x7f79a410d130 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 --2- 172.21.15.6:0/2010323368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4100c50 0x7f79a4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 --2- 172.21.15.6:0/2010323368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 0x7f79a4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 -- 172.21.15.6:0/2010323368 >> 172.21.15.6:0/2010323368 conn(0x7f79a40774f0 msgr2=0x7f79a40778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 -- 172.21.15.6:0/2010323368 shutdown_connections 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.186+0000 7f79a95ff700 1 -- 172.21.15.6:0/2010323368 wait complete. 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 Processor -- start 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 -- start start 2023-11-17T00:10:55.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 0x7f79a41969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 0x7f79a419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419b480 con 0x7f79a4100c50 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419b5f0 con 0x7f79a41015f0 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.187+0000 7f79a95ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419b7b0 con 0x7f79a4105560 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58884/0 (socket says 172.21.15.6:58884) 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1417795380 learned_addr learned my addr 172.21.15.6:0/1417795380 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a37fe700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 0x7f79a419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f799bfff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 0x7f79a41969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1417795380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 msgr2=0x7f79a419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 0x7f79a419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1417795380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 msgr2=0x7f79a41969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 0x7f79a41969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1417795380 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009d00 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a37fe700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 0x7f79a419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.188+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f799400e9a0 tx=0x7f799400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.189+0000 7f79a0ff9700 1 -- 172.21.15.6:0/1417795380 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994009790 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.189+0000 7f79a0ff9700 1 -- 172.21.15.6:0/1417795380 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7994010430 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.189+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a4110360 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.189+0000 7f79a0ff9700 1 -- 172.21.15.6:0/1417795380 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994018680 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.189+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41101d0 con 0x7f79a4100c50 2023-11-17T00:10:55.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.192+0000 7f79a0ff9700 1 -- 172.21.15.6:0/1417795380 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f79940105a0 con 0x7f79a4100c50 2023-11-17T00:10:55.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.193+0000 7f79a0ff9700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 0x7f79900a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.193+0000 7f79a0ff9700 1 -- 172.21.15.6:0/1417795380 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7994014070 con 0x7f79a4100c50 2023-11-17T00:10:55.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.194+0000 7f799bfff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 0x7f79900a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.194+0000 7f799bfff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 0x7f79900a37f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79a4197ab0 tx=0x7f798c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 Processor -- start 2023-11-17T00:10:55.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 -- start start 2023-11-17T00:10:55.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed6fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ec700e0 con 0x56429ed6abe0 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ecc2920 con 0x56429ed6b4f0 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ecbbdb0 con 0x56429ed6f930 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.288+0000 7f799bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58900/0 (socket says 172.21.15.6:58900) 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 -- 172.21.15.6:0/4038274670 learned_addr learned my addr 172.21.15.6:0/4038274670 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a37fe700 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed6fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 -- 172.21.15.6:0/4038274670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 msgr2=0x56429ed6fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed6fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 -- 172.21.15.6:0/4038274670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 msgr2=0x56429ed6f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed6f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 -- 172.21.15.6:0/4038274670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429eb3d6b0 con 0x56429ed6abe0 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f79940969d0 tx=0x7f7994099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7e6d8b4c9d8290 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79997fa700 1 -- 172.21.15.6:0/4038274670 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799409df90 con 0x56429ed6abe0 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79997fa700 1 -- 172.21.15.6:0/4038274670 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79940a23b0 con 0x56429ed6abe0 2023-11-17T00:10:55.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.289+0000 7f79997fa700 1 -- 172.21.15.6:0/4038274670 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994099700 con 0x56429ed6abe0 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 -- 172.21.15.6:0/4038274670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 msgr2=0x56429ed6afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f79940969d0 tx=0x7f7994099990 comp rx=0 tx=0).stop 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 -- 172.21.15.6:0/4038274670 shutdown_connections 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6abe0 0x56429ed6afb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/4038274670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 -- 172.21.15.6:0/4038274670 >> 172.21.15.6:0/4038274670 conn(0x56429ed603e0 msgr2=0x56429ed65bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.290+0000 7f79bcf1db80 1 -- 172.21.15.6:0/4038274670 shutdown_connections 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.291+0000 7f79bcf1db80 1 -- 172.21.15.6:0/4038274670 wait complete. 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.291+0000 7f79bcf1db80 1 -- start start 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.291+0000 7f79bcf1db80 1 Processor -- start 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ec5d960 con 0x56429ed6f930 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ec5d600 con 0x56429ed6b4f0 2023-11-17T00:10:55.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429eb3f9a0 con 0x56429ed6abe0 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.292+0000 7f79bcf1db80 10 client.0 _ll_register_callbacks cb 0x56429ec7b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f799bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:56672/0 (socket says 172.21.15.6:56672) 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1869546695 learned_addr learned my addr 172.21.15.6:0/1869546695 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 msgr2=0x56429ed743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1869546695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 msgr2=0x56429ed74ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed74ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 -- 172.21.15.6:0/1869546695 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41101d0 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79940cab60 tx=0x7f7994010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46545c2b7fc5db9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.293+0000 7f798a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994099660 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f79940a1f20 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56429ed5dba0 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79940a20e0 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 10 client.25978 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56429ec82210 con 0x56429ed6abe0 2023-11-17T00:10:55.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56429ed5dba0 con 0x56429ed6abe0 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7994098ec0 con 0x56429ed6abe0 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 10 client.25978 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 20 client.25978 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429ed84580 0x56429ed86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 4 client.25978 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:55.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.294+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56429ec7e120 con 0x56429ed84580 2023-11-17T00:10:55.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.295+0000 7f799bfff700 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429ed84580 0x56429ed86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.295+0000 7f799bfff700 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429ed84580 0x56429ed86990 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f810224bf640013 server_cookie=41b864bedadbe493 in_seq=0 out_seq=0 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.295+0000 7f798a7fc700 10 client.25978 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f798a7fc700 1 -- 172.21.15.6:0/1869546695 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56429ec7e120 con 0x56429ed84580 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f798a7fc700 10 client.25978 handle_command_reply: tid=0 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 client.25978 shutdown 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 msgr2=0x56429ed77190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79940cab60 tx=0x7f7994010ab0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 shutdown_connections 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429ed84580 0x56429ed86990 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed74ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6abe0 0x56429ed77190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/1869546695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6b4f0 0x56429ed743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f798a7fc700 0 client.25978 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:55.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 >> 172.21.15.6:0/1869546695 conn(0x56429ed603e0 msgr2=0x56429ed70de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.296+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 shutdown_connections 2023-11-17T00:10:55.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.297+0000 7f79bcf1db80 1 -- 172.21.15.6:0/1869546695 wait complete. 2023-11-17T00:10:55.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.297+0000 7f79bcf1db80 20 client.25978 trim_cache size 0 max 16384 2023-11-17T00:10:55.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 Processor -- start 2023-11-17T00:10:55.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 -- start start 2023-11-17T00:10:55.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed74350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6b4f0 0x56429ed734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ed73a20 con 0x56429ed6f930 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ed6e2e0 con 0x56429ed6abe0 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ed6e530 con 0x56429ed6b4f0 2023-11-17T00:10:55.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.315+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed74350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58928/0 (socket says 172.21.15.6:58928) 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 -- 172.21.15.6:0/2696509328 learned_addr learned my addr 172.21.15.6:0/2696509328 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f799bfff700 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6b4f0 0x56429ed734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 -- 172.21.15.6:0/2696509328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6b4f0 msgr2=0x56429ed734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6b4f0 0x56429ed734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 -- 172.21.15.6:0/2696509328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 msgr2=0x56429ed74350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed74350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 -- 172.21.15.6:0/2696509328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429ed61b80 con 0x56429ed6f930 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f79a37fe700 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f799c009390 tx=0x7f799c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9acdceae1078cb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.316+0000 7f7999ffb700 1 -- 172.21.15.6:0/2696509328 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c021070 con 0x56429ed6f930 2023-11-17T00:10:55.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f7999ffb700 1 -- 172.21.15.6:0/2696509328 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f799c009460 con 0x56429ed6f930 2023-11-17T00:10:55.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f7999ffb700 1 -- 172.21.15.6:0/2696509328 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c00e040 con 0x56429ed6f930 2023-11-17T00:10:55.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2696509328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 msgr2=0x56429ed65830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f799c009390 tx=0x7f799c0093c0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2696509328 shutdown_connections 2023-11-17T00:10:55.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6f930 0x56429ed65830 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed74350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2696509328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6b4f0 0x56429ed734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2696509328 >> 172.21.15.6:0/2696509328 conn(0x56429ed603e0 msgr2=0x56429ed73c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2696509328 shutdown_connections 2023-11-17T00:10:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.317+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2696509328 wait complete. 2023-11-17T00:10:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.318+0000 7f79bcf1db80 1 -- start start 2023-11-17T00:10:55.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.318+0000 7f79bcf1db80 1 Processor -- start 2023-11-17T00:10:55.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.318+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.318+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6b4f0 0x56429ed79e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.318+0000 7f79bcf1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed7a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ed7aa50 con 0x56429ed6b4f0 2023-11-17T00:10:55.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ed7abc0 con 0x56429ed6abe0 2023-11-17T00:10:55.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79bcf1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429edd8410 con 0x56429ed6f930 2023-11-17T00:10:55.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79bcf1db80 10 client.0 _ll_register_callbacks cb 0x7f798c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:55.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f799bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6b4f0 0x56429ed79e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40872/0 (socket says 172.21.15.6:40872) 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 -- 172.21.15.6:0/2306702154 learned_addr learned my addr 172.21.15.6:0/2306702154 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a37fe700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed7a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 -- 172.21.15.6:0/2306702154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 msgr2=0x56429ed7a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed7a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 -- 172.21.15.6:0/2306702154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6b4f0 msgr2=0x56429ed79e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:55.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6b4f0 0x56429ed79e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 -- 172.21.15.6:0/2306702154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a410d940 con 0x56429ed6abe0 2023-11-17T00:10:55.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a37fe700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed7a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:55.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f79a2ffd700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f79940998e0 tx=0x7f7994094d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d634720cc1890af8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:55.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.319+0000 7f798a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:55.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940962a0 con 0x56429ed6abe0 2023-11-17T00:10:55.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7994099f60 con 0x56429ed6abe0 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79940a23b0 con 0x56429ed6abe0 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7994099660 con 0x56429ed6abe0 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f79bcf1db80 10 client.26003 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56429eae9e60 con 0x56429ed6abe0 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79940a23b0 con 0x56429ed6abe0 2023-11-17T00:10:55.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.320+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7994083240 con 0x56429ed6abe0 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 10 client.26003 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 20 client.26003 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429edd95a0 0x56429eddb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 4 client.26003 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56429ec8a6e0 con 0x56429edd95a0 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f799bfff700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429edd95a0 0x56429eddb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:55.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f799bfff700 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429edd95a0 0x56429eddb980 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6105e0330d6e9382 server_cookie=c042d7164765c40f in_seq=0 out_seq=0 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.321+0000 7f798a7fc700 10 client.26003 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f798a7fc700 1 -- 172.21.15.6:0/2306702154 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56429ec8a6e0 con 0x56429edd95a0 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f798a7fc700 10 client.26003 handle_command_reply: tid=0 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 client.26003 shutdown 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 msgr2=0x56429ed7bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f79940998e0 tx=0x7f7994094d80 comp rx=0 tx=0).stop 2023-11-17T00:10:55.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 shutdown_connections 2023-11-17T00:10:55.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429ed6f930 0x56429ed7a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56429ed6abe0 0x56429ed7bab0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56429edd95a0 0x56429eddb980 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 --2- 172.21.15.6:0/2306702154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56429ed6b4f0 0x56429ed79e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 >> 172.21.15.6:0/2306702154 conn(0x56429ed603e0 msgr2=0x56429ed6e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f798a7fc700 0 client.26003 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 shutdown_connections 2023-11-17T00:10:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.323+0000 7f79bcf1db80 1 -- 172.21.15.6:0/2306702154 wait complete. 2023-11-17T00:10:55.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79bcf1db80 20 client.26003 trim_cache size 0 max 16384 2023-11-17T00:10:55.772 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:55.772 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:10:55.772 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "2e3110be-aef7-4a0c-a049-11c9f6bec6ff", 2023-11-17T00:10:55.772 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:10:55.773 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 msgr2=0x7f79900a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 0x7f79900a37f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79a4197ab0 tx=0x7f798c009260 comp rx=0 tx=0).stop 2023-11-17T00:10:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 msgr2=0x7f79a41964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f799400e9a0 tx=0x7f799400ecb0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 shutdown_connections 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f79900a1340 0x7f79900a37f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79a4100c50 0x7f79a41964a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79a4105560 0x7f79a419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 --2- 172.21.15.6:0/1417795380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79a41015f0 0x7f79a41969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 >> 172.21.15.6:0/1417795380 conn(0x7f79a40774f0 msgr2=0x7f79a4103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.324+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 shutdown_connections 2023-11-17T00:10:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:55.325+0000 7f79a95ff700 1 -- 172.21.15.6:0/1417795380 wait complete. 2023-11-17T00:10:55.778 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:10:55.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:55.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 85 -lt 100 ']' 2023-11-17T00:10:55.780 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 85 + 1 2023-11-17T00:10:55.781 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=86 2023-11-17T00:10:55.781 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:56.324 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:56.324 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:56.377 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:56.377 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 86 -lt 100 ']' 2023-11-17T00:10:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 86 + 1 2023-11-17T00:10:56.447 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=87 2023-11-17T00:10:56.447 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:56.509 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:56.512 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:56.512 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:56.512 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:10:56.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:10:56.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: pgmap v64: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-11-17T00:10:56.724 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:56 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: pgmap v64: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-11-17T00:10:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:56 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:10:56.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:10:56.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:10:56.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:10:56.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: pgmap v64: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-11-17T00:10:56.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:56 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:10:57.049 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:57.050 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:57.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:57.267 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:57.277 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:57.277 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:57.277 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 87 -lt 100 ']' 2023-11-17T00:10:57.277 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 87 + 1 2023-11-17T00:10:57.314 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=88 2023-11-17T00:10:57.314 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:57.387 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:57.388 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:57.966 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:57.966 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:58.141 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:58.142 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:58.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:58.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:58.161 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 88 -lt 100 ']' 2023-11-17T00:10:58.161 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 88 + 1 2023-11-17T00:10:58.206 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=89 2023-11-17T00:10:58.207 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:58.283 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:58.283 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:58.286 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:58.286 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:58.287 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:58.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 -- 172.21.15.6:0/931557199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 msgr2=0x7f1e00101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 --2- 172.21.15.6:0/931557199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00101020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1df00099c0 tx=0x7f1df0009cd0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 -- 172.21.15.6:0/931557199 shutdown_connections 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 --2- 172.21.15.6:0/931557199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00101020 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 --2- 172.21.15.6:0/931557199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00105560 0x7f1e0010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 --2- 172.21.15.6:0/931557199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e001015f0 0x7f1e00104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 -- 172.21.15.6:0/931557199 >> 172.21.15.6:0/931557199 conn(0x7f1e000774f0 msgr2=0x7f1e000778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 -- 172.21.15.6:0/931557199 shutdown_connections 2023-11-17T00:10:58.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.468+0000 7f1e057ff700 1 -- 172.21.15.6:0/931557199 wait complete. 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.469+0000 7f1e057ff700 1 Processor -- start 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 -- start start 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e001015f0 0x7f1e00198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00105560 0x7f1e0019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019d5e0 con 0x7f1e00100c50 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019d750 con 0x7f1e001015f0 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1e057ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019d910 con 0x7f1e00105560 2023-11-17T00:10:58.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58954/0 (socket says 172.21.15.6:58954) 2023-11-17T00:10:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1dfeffd700 1 -- 172.21.15.6:0/723946579 learned_addr learned my addr 172.21.15.6:0/723946579 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e001015f0 0x7f1e00198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.470+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00105560 0x7f1e0019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 -- 172.21.15.6:0/723946579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00105560 msgr2=0x7f1e0019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00105560 0x7f1e0019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 -- 172.21.15.6:0/723946579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e001015f0 msgr2=0x7f1e00198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e001015f0 0x7f1e00198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 -- 172.21.15.6:0/723946579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00095f0 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b580 tx=0x7f1df000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1de7fff700 1 -- 172.21.15.6:0/723946579 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df001d070 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1de7fff700 1 -- 172.21.15.6:0/723946579 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0005050 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1de7fff700 1 -- 172.21.15.6:0/723946579 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0004a40 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e00110360 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.471+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001101d0 con 0x7f1e00100c50 2023-11-17T00:10:58.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.474+0000 7f1de7fff700 1 -- 172.21.15.6:0/723946579 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1df0005430 con 0x7f1e00100c50 2023-11-17T00:10:58.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.476+0000 7f1de7fff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 0x7f1de80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.476+0000 7f1de7fff700 1 -- 172.21.15.6:0/723946579 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1df00cdda0 con 0x7f1e00100c50 2023-11-17T00:10:58.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.476+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 0x7f1de80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.476+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 0x7f1de80a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e00199c10 tx=0x7f1df40073d0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.554+0000 7f1e1911eb80 1 Processor -- start 2023-11-17T00:10:58.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.554+0000 7f1e1911eb80 1 -- start start 2023-11-17T00:10:58.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.554+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb19370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 0x562f7fb19d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fa1a060 con 0x562f7fb14b60 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fa6c8a0 con 0x562f7fb15470 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fa65d30 con 0x562f7fb198b0 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58970/0 (socket says 172.21.15.6:58970) 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 -- 172.21.15.6:0/1296996861 learned_addr learned my addr 172.21.15.6:0/1296996861 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb19370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 0x562f7fb19d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 -- 172.21.15.6:0/1296996861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 msgr2=0x562f7fb19d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 0x562f7fb19d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 -- 172.21.15.6:0/1296996861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 msgr2=0x562f7fb19370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb19370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dfeffd700 1 -- 172.21.15.6:0/1296996861 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f7f8e7630 con 0x562f7fb14b60 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.555+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 0x562f7fb19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1df00a2a40 tx=0x7f1df009abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4411f92adfb41234 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1de57fa700 1 -- 172.21.15.6:0/1296996861 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0098f50 con 0x562f7fb14b60 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1de57fa700 1 -- 172.21.15.6:0/1296996861 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0004c20 con 0x562f7fb14b60 2023-11-17T00:10:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1de57fa700 1 -- 172.21.15.6:0/1296996861 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df009a2c0 con 0x562f7fb14b60 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1e1911eb80 1 -- 172.21.15.6:0/1296996861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 msgr2=0x562f7fb14f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.556+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1df00a2a40 tx=0x7f1df009abb0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 -- 172.21.15.6:0/1296996861 shutdown_connections 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb14b60 0x562f7fb14f30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb198b0 0x562f7fb19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/1296996861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb19370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 -- 172.21.15.6:0/1296996861 >> 172.21.15.6:0/1296996861 conn(0x562f7fb0a360 msgr2=0x562f7fb0fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 -- 172.21.15.6:0/1296996861 shutdown_connections 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 -- 172.21.15.6:0/1296996861 wait complete. 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.557+0000 7f1e1911eb80 1 -- start start 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 Processor -- start 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb20f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fa078e0 con 0x562f7fb198b0 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fa07580 con 0x562f7fb15470 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.558+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7f8e9920 con 0x562f7fb14b60 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1e1911eb80 10 client.0 _ll_register_callbacks cb 0x562f7fa25780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb20f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58986/0 (socket says 172.21.15.6:58986) 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 -- 172.21.15.6:0/3811860280 learned_addr learned my addr 172.21.15.6:0/3811860280 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 -- 172.21.15.6:0/3811860280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 msgr2=0x562f7fb20f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb20f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 -- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 msgr2=0x562f7fb1e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb1e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 -- 172.21.15.6:0/3811860280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001101d0 con 0x562f7fb198b0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007c60 tx=0x7f1df8007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5796b2ef708c02 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.559+0000 7f1de2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800e040 con 0x562f7fb198b0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f7fb07b20 con 0x562f7fb198b0 2023-11-17T00:10:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1df8013970 con 0x562f7fb198b0 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df8013b30 con 0x562f7fb198b0 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1e1911eb80 10 client.15954 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562f7fa2c190 con 0x562f7fb198b0 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f7fb07b20 con 0x562f7fb198b0 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1df801e3b0 con 0x562f7fb198b0 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1e1911eb80 10 client.15954 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.560+0000 7f1e1911eb80 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1e1911eb80 20 client.15954 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb2e5d0 0x562f7fb309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1e1911eb80 4 client.15954 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f7fa280a0 con 0x562f7fb2e5d0 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb2e5d0 0x562f7fb309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb2e5d0 0x562f7fb309e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402d23742631e3f5 server_cookie=2a48f4070df6cdb2 in_seq=0 out_seq=0 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.561+0000 7f1de2ff5700 10 client.15954 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.562+0000 7f1de2ff5700 1 -- 172.21.15.6:0/3811860280 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f7fa280a0 con 0x562f7fb2e5d0 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.562+0000 7f1de2ff5700 10 client.15954 handle_command_reply: tid=0 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.562+0000 7f1e1911eb80 1 client.15954 shutdown 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.562+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 msgr2=0x562f7fb1e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.562+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007c60 tx=0x7f1df8007c90 comp rx=0 tx=0).stop 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 shutdown_connections 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb2e5d0 0x562f7fb309e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb1e6f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3811860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb1e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 >> 172.21.15.6:0/3811860280 conn(0x562f7fb0a360 msgr2=0x562f7fb1ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1de2ff5700 0 client.15954 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 shutdown_connections 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3811860280 wait complete. 2023-11-17T00:10:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.563+0000 7f1e1911eb80 20 client.15954 trim_cache size 0 max 16384 2023-11-17T00:10:58.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.582+0000 7f1e1911eb80 1 Processor -- start 2023-11-17T00:10:58.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.582+0000 7f1e1911eb80 1 -- start start 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 0x562f7fb0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb15470 0x562f7fb25750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb2af60 con 0x562f7fb198b0 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb11370 con 0x562f7fb14b60 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb18140 con 0x562f7fb15470 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 0x562f7fb0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dfe7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb15470 0x562f7fb25750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58992/0 (socket says 172.21.15.6:58992) 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dff7fe700 1 -- 172.21.15.6:0/3645932758 learned_addr learned my addr 172.21.15.6:0/3645932758 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:58.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dfeffd700 1 -- 172.21.15.6:0/3645932758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb15470 msgr2=0x562f7fb25750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.583+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb15470 0x562f7fb25750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1dfeffd700 1 -- 172.21.15.6:0/3645932758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 msgr2=0x562f7fb20ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1dfeffd700 1 -- 172.21.15.6:0/3645932758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f7fa2bee0 con 0x562f7fb14b60 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 0x562f7fb0fc80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x562f7fb1a300 tx=0x7f1df009a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e477fbbd70a8e3e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1de5ffb700 1 -- 172.21.15.6:0/3645932758 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0004d90 con 0x562f7fb14b60 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1de5ffb700 1 -- 172.21.15.6:0/3645932758 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0098f50 con 0x562f7fb14b60 2023-11-17T00:10:58.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.584+0000 7f1de5ffb700 1 -- 172.21.15.6:0/3645932758 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00a2ae0 con 0x562f7fb14b60 2023-11-17T00:10:58.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3645932758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 msgr2=0x562f7fb0fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 0x562f7fb0fc80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x562f7fb1a300 tx=0x7f1df009a5c0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3645932758 shutdown_connections 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb20ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb14b60 0x562f7fb0fc80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/3645932758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb15470 0x562f7fb25750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3645932758 >> 172.21.15.6:0/3645932758 conn(0x562f7fb0a360 msgr2=0x562f7fb2a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3645932758 shutdown_connections 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- 172.21.15.6:0/3645932758 wait complete. 2023-11-17T00:10:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.585+0000 7f1e1911eb80 1 -- start start 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 Processor -- start 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb81450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb24050 con 0x562f7fb198b0 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb81990 con 0x562f7fb15470 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7fb81b00 con 0x562f7fb14b60 2023-11-17T00:10:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1e1911eb80 10 client.0 _ll_register_callbacks cb 0x7f1df80075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:10:58.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1dfe7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb81450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1dfe7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40924/0 (socket says 172.21.15.6:40924) 2023-11-17T00:10:58.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.586+0000 7f1dfe7fc700 1 -- 172.21.15.6:0/2192105900 learned_addr learned my addr 172.21.15.6:0/2192105900 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:10:58.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 -- 172.21.15.6:0/2192105900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 msgr2=0x562f7fb23220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb23220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 -- 172.21.15.6:0/2192105900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 msgr2=0x562f7fb81450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb81450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 -- 172.21.15.6:0/2192105900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f7fa2c190 con 0x562f7fb15470 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dff7fe700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb81450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1dfe7fc700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006a2c0 tx=0x7f1df40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=780dbb5dbadb0690 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1de2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:10:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4026070 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1df4017030 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.587+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df0004c20 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4003950 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 10 client.26021 fetch_fsmap learned FSMap version 20 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562f7f8e7630 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0004c20 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1df4011c50 con 0x562f7fb15470 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 10 client.26021 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 20 client.26021 populate_metadata read hostname 'smithi006' 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb83310 0x562f7fb85720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 4 client.26021 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:10:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f7fa34660 con 0x562f7fb83310 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.588+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb83310 0x562f7fb85720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1dfeffd700 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb83310 0x562f7fb85720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc9fe6c79977e42 server_cookie=ad779f2f5b0c115b in_seq=0 out_seq=0 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1de2ff5700 10 client.26021 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1de2ff5700 1 -- 172.21.15.6:0/2192105900 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f7fa34660 con 0x562f7fb83310 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1de2ff5700 10 client.26021 handle_command_reply: tid=0 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 client.26021 shutdown 2023-11-17T00:10:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 msgr2=0x562f7fb23760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006a2c0 tx=0x7f1df40047c0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 shutdown_connections 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562f7fb83310 0x562f7fb85720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562f7fb198b0 0x562f7fb81450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562f7fb14b60 0x562f7fb23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 --2- 172.21.15.6:0/2192105900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f7fb15470 0x562f7fb23760 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1de2ff5700 0 client.26021 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.589+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 >> 172.21.15.6:0/2192105900 conn(0x562f7fb0a360 msgr2=0x562f7fb13960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 shutdown_connections 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e1911eb80 1 -- 172.21.15.6:0/2192105900 wait complete. 2023-11-17T00:10:58.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e1911eb80 20 client.26021 trim_cache size 0 max 16384 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 msgr2=0x7f1de80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 0x7f1de80a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e00199c10 tx=0x7f1df40073d0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 msgr2=0x7f1e00198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b580 tx=0x7f1df000f690 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 shutdown_connections 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1de80a1340 0x7f1de80a37f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e00100c50 0x7f1e00198600 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e00105560 0x7f1e0019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 --2- 172.21.15.6:0/723946579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e001015f0 0x7f1e00198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 >> 172.21.15.6:0/723946579 conn(0x7f1e000774f0 msgr2=0x7f1e000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.590+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 shutdown_connections 2023-11-17T00:10:58.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:10:58.591+0000 7f1e057ff700 1 -- 172.21.15.6:0/723946579 wait complete. 2023-11-17T00:10:58.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:10:58.677 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:10:58.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:10:58 smithi006 ceph-mon[97840]: pgmap v65: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-17T00:10:58.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:10:58 smithi087 ceph-mon[131182]: pgmap v65: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-17T00:10:58.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:10:58 smithi186 ceph-mon[131487]: pgmap v65: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-17T00:10:58.968 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:58.972 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:59.044 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:59.044 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:59.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:59.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:59.068 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 89 -lt 100 ']' 2023-11-17T00:10:59.068 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 89 + 1 2023-11-17T00:10:59.122 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=90 2023-11-17T00:10:59.122 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:59.128 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:59.129 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:59.132 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:59.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:59.133 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:10:59.748 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:59.748 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:59.832 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:10:59.833 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:10:59.877 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:59.878 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:10:59.878 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 90 -lt 100 ']' 2023-11-17T00:10:59.878 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 90 + 1 2023-11-17T00:10:59.921 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=91 2023-11-17T00:10:59.922 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:10:59.975 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:10:59.975 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:10:59.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:10:59.979 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:10:59.979 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:00.590 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:00.590 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:00.726 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:00 smithi006 ceph-mon[97840]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-11-17T00:11:00.769 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:00.769 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:00.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:00.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:00.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 91 -lt 100 ']' 2023-11-17T00:11:00.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 91 + 1 2023-11-17T00:11:00.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:00 smithi087 ceph-mon[131182]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-11-17T00:11:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:00 smithi186 ceph-mon[131487]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-11-17T00:11:00.888 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=92 2023-11-17T00:11:00.888 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:00.951 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:00.951 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:00.952 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:00.953 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:00.953 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:01.925 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:01.926 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:01.935 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:01.935 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:01.935 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 92 -lt 100 ']' 2023-11-17T00:11:01.935 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 92 + 1 2023-11-17T00:11:01.979 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=93 2023-11-17T00:11:01.980 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:02.034 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:02.034 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:02.036 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:02.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:02.038 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:02.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 -- 172.21.15.6:0/3493418016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 msgr2=0x7f4c14101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:02.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 --2- 172.21.15.6:0/3493418016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14101090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009920 tx=0x7f4c08009c30 comp rx=0 tx=0).stop 2023-11-17T00:11:02.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 -- 172.21.15.6:0/3493418016 shutdown_connections 2023-11-17T00:11:02.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 --2- 172.21.15.6:0/3493418016 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14105620 0x7f4c1410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 --2- 172.21.15.6:0/3493418016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14101090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 --2- 172.21.15.6:0/3493418016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14101660 0x7f4c14104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 -- 172.21.15.6:0/3493418016 >> 172.21.15.6:0/3493418016 conn(0x7f4c140774d0 msgr2=0x7f4c140778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:02.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 -- 172.21.15.6:0/3493418016 shutdown_connections 2023-11-17T00:11:02.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.251+0000 7f4c1afff700 1 -- 172.21.15.6:0/3493418016 wait complete. 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.252+0000 7f4c1afff700 1 Processor -- start 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.252+0000 7f4c1afff700 1 -- start start 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14105620 0x7f4c1419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419b530 con 0x7f4c14101660 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419b6a0 con 0x7f4c14100cc0 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419b860 con 0x7f4c14105620 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42982/0 (socket says 172.21.15.6:42982) 2023-11-17T00:11:02.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 -- 172.21.15.6:0/493825035 learned_addr learned my addr 172.21.15.6:0/493825035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c18d9b700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c1959c700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14105620 0x7f4c1419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 -- 172.21.15.6:0/493825035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14105620 msgr2=0x7f4c1419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14105620 0x7f4c1419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 -- 172.21.15.6:0/493825035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 msgr2=0x7f4c14196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.253+0000 7f4c13fff700 1 -- 172.21.15.6:0/493825035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009520 con 0x7f4c14101660 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c13fff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d980 tx=0x7f4c0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000f890 con 0x7f4c14101660 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c141103d0 con 0x7f4c14101660 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c00010460 con 0x7f4c14101660 2023-11-17T00:11:02.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000e610 con 0x7f4c14101660 2023-11-17T00:11:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.254+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c14110240 con 0x7f4c14101660 2023-11-17T00:11:02.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.255+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14110560 con 0x7f4c14101660 2023-11-17T00:11:02.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.257+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4c000105d0 con 0x7f4c14101660 2023-11-17T00:11:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.259+0000 7f4c11ffb700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.259+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4c000ccb30 con 0x7f4c14101660 2023-11-17T00:11:02.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.259+0000 7f4c18d9b700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:02.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.260+0000 7f4c18d9b700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 0x7f4c040a37f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800c010 tx=0x7f4c08019040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:02.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.266+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c00095b90 con 0x7f4c14101660 2023-11-17T00:11:02.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.524+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c14197940 con 0x7f4c14101660 2023-11-17T00:11:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.525+0000 7f4c11ffb700 1 -- 172.21.15.6:0/493825035 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+10736 (secure 0 0 0) 0x7f4c000991b0 con 0x7f4c14101660 2023-11-17T00:11:02.586 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:11:02.586 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:06:37.833900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6837/850260336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":850260336},{"type":"v1","addr":"172.21.15.87:6837","nonce":850260336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 msgr2=0x7f4c040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 0x7f4c040a37f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800c010 tx=0x7f4c08019040 comp rx=0 tx=0).stop 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 msgr2=0x7f4c14196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d980 tx=0x7f4c0000dc90 comp rx=0 tx=0).stop 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 shutdown_connections 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c040a1340 0x7f4c040a37f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c14105620 0x7f4c1419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c14100cc0 0x7f4c14196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 --2- 172.21.15.6:0/493825035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c14101660 0x7f4c14196a90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 >> 172.21.15.6:0/493825035 conn(0x7f4c140774d0 msgr2=0x7f4c140ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 shutdown_connections 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:02.528+0000 7f4c1afff700 1 -- 172.21.15.6:0/493825035 wait complete. 2023-11-17T00:11:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 20 2023-11-17T00:11:02.615 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:02.615 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:02.622 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:11:02.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:02 smithi006 ceph-mon[97840]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:02.785 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:02.785 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:02.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:02.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:02.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 93 -lt 100 ']' 2023-11-17T00:11:02.810 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 93 + 1 2023-11-17T00:11:02.838 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=94 2023-11-17T00:11:02.838 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:02.846 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:02.847 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:02.847 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:02.847 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:02.848 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:02.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:02 smithi087 ceph-mon[131182]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:02.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:02 smithi186 ceph-mon[131487]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:03.264 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:03.265 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:03.407 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:03.408 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:03.435 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:03.435 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:03.435 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 94 -lt 100 ']' 2023-11-17T00:11:03.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 94 + 1 2023-11-17T00:11:03.471 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=95 2023-11-17T00:11:03.471 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:03.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:03.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:03.507 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:03.509 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:03.509 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:03.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:03 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/493825035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:03.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:03 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/493825035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:03.908 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:03 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/493825035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:04.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:04.131 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:04.310 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:04.311 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:04.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:04.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:04.318 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 95 -lt 100 ']' 2023-11-17T00:11:04.319 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 95 + 1 2023-11-17T00:11:04.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=96 2023-11-17T00:11:04.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:04.600 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:04.601 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:04.603 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:04.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:04.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:04.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:04 smithi087 ceph-mon[131182]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:04 smithi186 ceph-mon[131487]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:04 smithi006 ceph-mon[97840]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-11-17T00:11:05.315 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:05.315 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:05.574 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:05.574 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:05.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:05.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:05.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 96 -lt 100 ']' 2023-11-17T00:11:05.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 96 + 1 2023-11-17T00:11:05.779 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=97 2023-11-17T00:11:05.779 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 -- 172.21.15.6:0/670793030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec067dc0 msgr2=0x7f7bec068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/670793030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec067dc0 0x7f7bec068230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40099c0 tx=0x7f7bd4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 -- 172.21.15.6:0/670793030 shutdown_connections 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/670793030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec068770 0x7f7bec10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/670793030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 0x7f7bec105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/670793030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec067dc0 0x7f7bec068230 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 -- 172.21.15.6:0/670793030 >> 172.21.15.6:0/670793030 conn(0x7f7bec0763a0 msgr2=0x7f7bec0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 -- 172.21.15.6:0/670793030 shutdown_connections 2023-11-17T00:11:05.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.784+0000 7f7bf11ff700 1 -- 172.21.15.6:0/670793030 wait complete. 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.785+0000 7f7bf11ff700 1 Processor -- start 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.785+0000 7f7bf11ff700 1 -- start start 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.785+0000 7f7bf11ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.785+0000 7f7bf11ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec068770 0x7f7bec100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bf11ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 0x7f7bec1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bf11ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1008b0 con 0x7f7bec068770 2023-11-17T00:11:05.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bf11ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1a4b10 con 0x7f7bec1052a0 2023-11-17T00:11:05.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bf11ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1a4c50 con 0x7f7bec067dc0 2023-11-17T00:11:05.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec068770 0x7f7bec100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:47100/0 (socket says 172.21.15.6:47100) 2023-11-17T00:11:05.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 -- 172.21.15.6:0/3729026764 learned_addr learned my addr 172.21.15.6:0/3729026764 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:05.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7beb59e700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 0x7f7bec1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 -- 172.21.15.6:0/3729026764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 msgr2=0x7f7bec1a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 0x7f7bec1a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 -- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec068770 msgr2=0x7f7bec100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec068770 0x7f7bec100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.786+0000 7f7bead9d700 1 -- 172.21.15.6:0/3729026764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40095f0 con 0x7f7bec067dc0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7bead9d700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9a0 tx=0x7f7bdc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7be8d99700 1 -- 172.21.15.6:0/3729026764 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc009790 con 0x7f7bec067dc0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec110390 con 0x7f7bec067dc0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7be8d99700 1 -- 172.21.15.6:0/3729026764 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bdc00cbd0 con 0x7f7bec067dc0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7be8d99700 1 -- 172.21.15.6:0/3729026764 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc018680 con 0x7f7bec067dc0 2023-11-17T00:11:05.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.787+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec110200 con 0x7f7bec067dc0 2023-11-17T00:11:05.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.790+0000 7f7be8d99700 1 -- 172.21.15.6:0/3729026764 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7bdc00cd40 con 0x7f7bec067dc0 2023-11-17T00:11:05.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.791+0000 7f7be8d99700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 0x7f7bd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.792+0000 7f7be8d99700 1 -- 172.21.15.6:0/3729026764 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7bdc014070 con 0x7f7bec067dc0 2023-11-17T00:11:05.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.792+0000 7f7be3fff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 0x7f7bd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.792+0000 7f7be3fff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 0x7f7bd80a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:05.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:05.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:05.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:05.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:05.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.911+0000 7f7c04a91b80 1 Processor -- start 2023-11-17T00:11:05.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.911+0000 7f7c04a91b80 1 -- start start 2023-11-17T00:11:05.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b331f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b336d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b237060 con 0x55c26b332470 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b2898a0 con 0x55c26b331b60 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b282d30 con 0x55c26b3368b0 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43010/0 (socket says 172.21.15.6:43010) 2023-11-17T00:11:05.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 -- 172.21.15.6:0/1878707353 learned_addr learned my addr 172.21.15.6:0/1878707353 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7bead9d700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b331f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7beb59e700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b336d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 -- 172.21.15.6:0/1878707353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 msgr2=0x55c26b336d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b336d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 -- 172.21.15.6:0/1878707353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 msgr2=0x55c26b331f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b331f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.912+0000 7f7be3fff700 1 -- 172.21.15.6:0/1878707353 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c26b104630 con 0x55c26b332470 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.913+0000 7f7beb59e700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b336d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:05.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.913+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4015000 tx=0x7f7bd401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206ab5fd79f9b6ae server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.913+0000 7f7be17fa700 1 -- 172.21.15.6:0/1878707353 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400b990 con 0x55c26b332470 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.913+0000 7f7be17fa700 1 -- 172.21.15.6:0/1878707353 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd401e040 con 0x55c26b332470 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.913+0000 7f7be17fa700 1 -- 172.21.15.6:0/1878707353 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40306e0 con 0x55c26b332470 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1878707353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 msgr2=0x55c26b336370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4015000 tx=0x7f7bd401f000 comp rx=0 tx=0).stop 2023-11-17T00:11:05.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1878707353 shutdown_connections 2023-11-17T00:11:05.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b336d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b331f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1878707353 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b336370 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.914+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1878707353 >> 172.21.15.6:0/1878707353 conn(0x55c26b327360 msgr2=0x55c26b32cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.915+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1878707353 shutdown_connections 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.915+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1878707353 wait complete. 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.915+0000 7f7c04a91b80 1 -- start start 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.915+0000 7f7c04a91b80 1 Processor -- start 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 0x55c26b33b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b2248e0 con 0x55c26b331b60 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b224580 con 0x55c26b332470 2023-11-17T00:11:05.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b106920 con 0x55c26b3368b0 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7c04a91b80 10 client.0 _ll_register_callbacks cb 0x55c26b242780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 0x55c26b33b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43018/0 (socket says 172.21.15.6:43018) 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 -- 172.21.15.6:0/563783994 learned_addr learned my addr 172.21.15.6:0/563783994 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7beb59e700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 -- 172.21.15.6:0/563783994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 msgr2=0x55c26b33b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 -- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 msgr2=0x55c26b33b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 0x55c26b33b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7bead9d700 1 -- 172.21.15.6:0/563783994 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec10d9d0 con 0x55c26b331b60 2023-11-17T00:11:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.916+0000 7f7be3fff700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 0x55c26b33b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bead9d700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc099f70 tx=0x7f7bdc099fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c321816c8da212 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bd27fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc0a2430 con 0x55c26b331b60 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7bdc0a1fa0 con 0x55c26b331b60 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c26b324b20 con 0x55c26b331b60 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bdc0a2160 con 0x55c26b331b60 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7c04a91b80 10 client.15978 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c26b249190 con 0x55c26b331b60 2023-11-17T00:11:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.917+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bdc0c8020 con 0x55c26b331b60 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bdc099060 con 0x55c26b331b60 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 20 client.15978 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b34b830 0x55c26b34dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 4 client.15978 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c26b2450a0 con 0x55c26b34b830 2023-11-17T00:11:05.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.918+0000 7f7be3fff700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b34b830 0x55c26b34dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.919+0000 7f7bd27fc700 10 client.15978 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.919+0000 7f7be3fff700 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b34b830 0x55c26b34dc10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607e33b978f449b2 server_cookie=a3faf530d207328 in_seq=0 out_seq=0 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7bd27fc700 1 -- 172.21.15.6:0/563783994 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c26b2450a0 con 0x55c26b34b830 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7bd27fc700 10 client.15978 handle_command_reply: tid=0 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 client.15978 shutdown 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 msgr2=0x55c26b33e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc099f70 tx=0x7f7bdc099fe0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 shutdown_connections 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b34b830 0x55c26b34dc10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b331b60 0x55c26b33e000 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7bd27fc700 0 client.15978 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/563783994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b332470 0x55c26b33b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 >> 172.21.15.6:0/563783994 conn(0x55c26b327360 msgr2=0x55c26b337d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 shutdown_connections 2023-11-17T00:11:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 1 -- 172.21.15.6:0/563783994 wait complete. 2023-11-17T00:11:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.920+0000 7f7c04a91b80 20 client.15978 trim_cache size 0 max 16384 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 Processor -- start 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 -- start start 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b33c8d0 con 0x55c26b332470 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b3351f0 con 0x55c26b331b60 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.934+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b34b690 con 0x55c26b3368b0 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43024/0 (socket says 172.21.15.6:43024) 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 -- 172.21.15.6:0/1983334374 learned_addr learned my addr 172.21.15.6:0/1983334374 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7bead9d700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7beb59e700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 -- 172.21.15.6:0/1983334374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 msgr2=0x55c26b33dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 -- 172.21.15.6:0/1983334374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 msgr2=0x55c26b342970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 -- 172.21.15.6:0/1983334374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c26b248ee0 con 0x55c26b332470 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7beb59e700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be3fff700 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82bdba00e1315f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.935+0000 7f7be1ffb700 1 -- 172.21.15.6:0/1983334374 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4036040 con 0x55c26b332470 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7be1ffb700 1 -- 172.21.15.6:0/1983334374 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd400b580 con 0x55c26b332470 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7be1ffb700 1 -- 172.21.15.6:0/1983334374 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400bb90 con 0x55c26b332470 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1983334374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 msgr2=0x55c26b347bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40044f0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1983334374 shutdown_connections 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b33dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/1983334374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b347bb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1983334374 >> 172.21.15.6:0/1983334374 conn(0x55c26b327360 msgr2=0x55c26b338c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1983334374 shutdown_connections 2023-11-17T00:11:05.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.936+0000 7f7c04a91b80 1 -- 172.21.15.6:0/1983334374 wait complete. 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.937+0000 7f7c04a91b80 1 -- start start 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.937+0000 7f7c04a91b80 1 Processor -- start 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.937+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b340ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b341570 con 0x55c26b332470 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b3416e0 con 0x55c26b331b60 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26b39e6c0 con 0x55c26b3368b0 2023-11-17T00:11:05.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7c04a91b80 10 client.0 _ll_register_callbacks cb 0x7f7bdc099960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43028/0 (socket says 172.21.15.6:43028) 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 -- 172.21.15.6:0/328828282 learned_addr learned my addr 172.21.15.6:0/328828282 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7beb59e700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b340ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7bead9d700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 -- 172.21.15.6:0/328828282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 msgr2=0x55c26b340ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b340ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 -- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 msgr2=0x55c26b342980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.938+0000 7f7be3fff700 1 -- 172.21.15.6:0/328828282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c26b104630 con 0x55c26b332470 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7be3fff700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4003d30 tx=0x7f7bd40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1b13dd8700d234 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:05.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7bd27fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4036070 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7bd403bb50 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bd401e040 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bd403bd10 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7c04a91b80 10 client.15984 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c26b099c60 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.939+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd401e040 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bd4044020 con 0x55c26b332470 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 20 client.15984 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b39f8b0 0x55c26b3a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:05.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 4 client.15984 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c26b251660 con 0x55c26b39f8b0 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7bead9d700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b39f8b0 0x55c26b3a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7bead9d700 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b39f8b0 0x55c26b3a1cc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937c739e5125ea25 server_cookie=a636b02487f44fbc in_seq=0 out_seq=0 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.940+0000 7f7bd27fc700 10 client.15984 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7bd27fc700 1 -- 172.21.15.6:0/328828282 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c26b251660 con 0x55c26b39f8b0 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7bd27fc700 10 client.15984 handle_command_reply: tid=0 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 client.15984 shutdown 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 msgr2=0x55c26b340960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4003d30 tx=0x7f7bd40043f0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 shutdown_connections 2023-11-17T00:11:05.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c26b3368b0 0x55c26b340ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c26b331b60 0x55c26b342980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c26b39f8b0 0x55c26b3a1cc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 --2- 172.21.15.6:0/328828282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c26b332470 0x55c26b340960 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 >> 172.21.15.6:0/328828282 conn(0x55c26b327360 msgr2=0x55c26b334930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7bd27fc700 0 client.15984 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 shutdown_connections 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 1 -- 172.21.15.6:0/328828282 wait complete. 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.941+0000 7f7c04a91b80 20 client.15984 trim_cache size 0 max 16384 2023-11-17T00:11:05.949 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:11:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 msgr2=0x7f7bd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 0x7f7bd80a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4005fb0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 msgr2=0x7f7bec0ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:05.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9a0 tx=0x7f7bdc00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 shutdown_connections 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7bd80a12c0 0x7f7bd80a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bec1052a0 0x7f7bec1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bec067dc0 0x7f7bec0ffb40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 --2- 172.21.15.6:0/3729026764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bec068770 0x7f7bec100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 >> 172.21.15.6:0/3729026764 conn(0x7f7bec0763a0 msgr2=0x7f7bec0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 shutdown_connections 2023-11-17T00:11:05.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:05.942+0000 7f7bf11ff700 1 -- 172.21.15.6:0/3729026764 wait complete. 2023-11-17T00:11:06.498 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:06.498 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:06.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:06 smithi006 ceph-mon[97840]: pgmap v69: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 KiB/s wr, 26 op/s 2023-11-17T00:11:06.749 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:06.749 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:06.760 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:06.760 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:06.760 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 97 -lt 100 ']' 2023-11-17T00:11:06.761 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 97 + 1 2023-11-17T00:11:06.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=98 2023-11-17T00:11:06.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:06.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:06 smithi087 ceph-mon[131182]: pgmap v69: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 KiB/s wr, 26 op/s 2023-11-17T00:11:06.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:06 smithi186 ceph-mon[131487]: pgmap v69: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 KiB/s wr, 26 op/s 2023-11-17T00:11:06.899 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:06.900 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:07.019 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:11:07.019 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 -- 172.21.15.6:0/3123315722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 msgr2=0x7f12f01016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 --2- 172.21.15.6:0/3123315722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01016b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0098f0 tx=0x7f12ec009c00 comp rx=0 tx=0).stop 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 -- 172.21.15.6:0/3123315722 shutdown_connections 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 --2- 172.21.15.6:0/3123315722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f010a800 0x7f12f010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 --2- 172.21.15.6:0/3123315722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01016b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 --2- 172.21.15.6:0/3123315722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f0101bf0 0x7f12f010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 -- 172.21.15.6:0/3123315722 >> 172.21.15.6:0/3123315722 conn(0x7f12f00763a0 msgr2=0x7f12f00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 -- 172.21.15.6:0/3123315722 shutdown_connections 2023-11-17T00:11:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.634+0000 7f12f7bff700 1 -- 172.21.15.6:0/3123315722 wait complete. 2023-11-17T00:11:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.676+0000 7f12f7bff700 1 Processor -- start 2023-11-17T00:11:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 -- start start 2023-11-17T00:11:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f0101bf0 0x7f12f0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f019b3b0 con 0x7f12f010a800 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f019b520 con 0x7f12f0101bf0 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f7bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f019b6e0 con 0x7f12f01012e0 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f619c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f619c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43038/0 (socket says 172.21.15.6:43038) 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f619c700 1 -- 172.21.15.6:0/368952433 learned_addr learned my addr 172.21.15.6:0/368952433 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f599b700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.677+0000 7f12f519a700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f0101bf0 0x7f12f0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 -- 172.21.15.6:0/368952433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 msgr2=0x7f12f01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 -- 172.21.15.6:0/368952433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f0101bf0 msgr2=0x7f12f0196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f0101bf0 0x7f12f0196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 -- 172.21.15.6:0/368952433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec009520 con 0x7f12f010a800 2023-11-17T00:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f619c700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ef30 tx=0x7f12e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12e6ffd700 1 -- 172.21.15.6:0/368952433 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8018070 con 0x7f12f010a800 2023-11-17T00:11:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12e6ffd700 1 -- 172.21.15.6:0/368952433 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12e800f040 con 0x7f12f010a800 2023-11-17T00:11:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f01102c0 con 0x7f12f010a800 2023-11-17T00:11:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12e6ffd700 1 -- 172.21.15.6:0/368952433 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800ccf0 con 0x7f12f010a800 2023-11-17T00:11:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.678+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f0110130 con 0x7f12f010a800 2023-11-17T00:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.681+0000 7f12e6ffd700 1 -- 172.21.15.6:0/368952433 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f12e8003a00 con 0x7f12f010a800 2023-11-17T00:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.683+0000 7f12e6ffd700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 0x7f12dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.683+0000 7f12e6ffd700 1 -- 172.21.15.6:0/368952433 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f12e80ccb20 con 0x7f12f010a800 2023-11-17T00:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.683+0000 7f12f599b700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 0x7f12dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.684+0000 7f12f599b700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 0x7f12dc0a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005e90 tx=0x7f12ec019040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:07.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 Processor -- start 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 -- start start 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431bb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da430bd000 con 0x55da430136b0 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4310f840 con 0x55da431c1b20 2023-11-17T00:11:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f130b492b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43108cd0 con 0x55da431c1f90 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43044/0 (socket says 172.21.15.6:43044) 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.828+0000 7f12f599b700 1 -- 172.21.15.6:0/4291404444 learned_addr learned my addr 172.21.15.6:0/4291404444 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f519a700 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431bb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f619c700 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 -- 172.21.15.6:0/4291404444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 msgr2=0x55da431bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 -- 172.21.15.6:0/4291404444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 msgr2=0x55da431bb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431bb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 -- 172.21.15.6:0/4291404444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da42f8a5d0 con 0x55da430136b0 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12f599b700 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0098c0 tx=0x7f12ec01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5bd55d4ebb49d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.829+0000 7f12dbfff700 1 -- 172.21.15.6:0/4291404444 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00b760 con 0x55da430136b0 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f12dbfff700 1 -- 172.21.15.6:0/4291404444 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ec01d040 con 0x55da430136b0 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f12dbfff700 1 -- 172.21.15.6:0/4291404444 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec02f6c0 con 0x55da430136b0 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 -- 172.21.15.6:0/4291404444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 msgr2=0x55da431c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0098c0 tx=0x7f12ec01e000 comp rx=0 tx=0).stop 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 -- 172.21.15.6:0/4291404444 shutdown_connections 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c15e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 --2- 172.21.15.6:0/4291404444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431bb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.830+0000 7f130b492b80 1 -- 172.21.15.6:0/4291404444 >> 172.21.15.6:0/4291404444 conn(0x55da431acf50 msgr2=0x55da431b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.831+0000 7f130b492b80 1 -- 172.21.15.6:0/4291404444 shutdown_connections 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.831+0000 7f130b492b80 1 -- 172.21.15.6:0/4291404444 wait complete. 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.831+0000 7f130b492b80 1 -- start start 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.831+0000 7f130b492b80 1 Processor -- start 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1b20 0x55da431cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da431c1f90 0x55da431c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da430aa880 con 0x55da431c1f90 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da430aa520 con 0x55da430136b0 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da42f8c8c0 con 0x55da431c1b20 2023-11-17T00:11:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f130b492b80 10 client.0 _ll_register_callbacks cb 0x55da430c8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f12f599b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f12f519a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1b20 0x55da431cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f12f619c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da431c1f90 0x55da431c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f12f599b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:39354/0 (socket says 172.21.15.6:39354) 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.832+0000 7f12f599b700 1 -- 172.21.15.6:0/2719055495 learned_addr learned my addr 172.21.15.6:0/2719055495 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 -- 172.21.15.6:0/2719055495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1b20 msgr2=0x55da431cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1b20 0x55da431cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 -- 172.21.15.6:0/2719055495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da431c1f90 msgr2=0x55da431c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da431c1f90 0x55da431c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 -- 172.21.15.6:0/2719055495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f010d950 con 0x55da430136b0 2023-11-17T00:11:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12f599b700 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12ec000c00 tx=0x7f12ec005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9787a615373569b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00ba30 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da431aa710 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f12ec00bb90 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.833+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12ec003680 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 10 client.26057 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55da430cf130 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da431aa710 con 0x55da430136b0 2023-11-17T00:11:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12ec02f900 con 0x55da430136b0 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 20 client.26057 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da431d4f60 0x55da431d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 4 client.26057 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da430cb040 con 0x55da431d4f60 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.834+0000 7f12f519a700 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da431d4f60 0x55da431d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.835+0000 7f12f519a700 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da431d4f60 0x55da431d7370 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1094f9c106e19cf server_cookie=920084dcf825e030 in_seq=0 out_seq=0 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.835+0000 7f12d97fa700 10 client.26057 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f12d97fa700 1 -- 172.21.15.6:0/2719055495 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da430cb040 con 0x55da431d4f60 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f12d97fa700 10 client.26057 handle_command_reply: tid=0 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 client.26057 shutdown 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 msgr2=0x55da431cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12ec000c00 tx=0x7f12ec005810 comp rx=0 tx=0).stop 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 shutdown_connections 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da431d4f60 0x55da431d7370 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da431c1f90 0x55da431c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da430136b0 0x55da431cae90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f12d97fa700 0 client.26057 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 --2- 172.21.15.6:0/2719055495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1b20 0x55da431cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 >> 172.21.15.6:0/2719055495 conn(0x55da431acf50 msgr2=0x55da431b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 shutdown_connections 2023-11-17T00:11:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.836+0000 7f130b492b80 1 -- 172.21.15.6:0/2719055495 wait complete. 2023-11-17T00:11:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.837+0000 7f130b492b80 20 client.26057 trim_cache size 0 max 16384 2023-11-17T00:11:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.852+0000 7f130b492b80 1 Processor -- start 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.852+0000 7f130b492b80 1 -- start start 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da431c5620 con 0x55da430136b0 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da431c65a0 con 0x55da431c1b20 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f130b492b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da431b2fb0 con 0x55da431c1f90 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43058/0 (socket says 172.21.15.6:43058) 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f12f599b700 1 -- 172.21.15.6:0/3982379397 learned_addr learned my addr 172.21.15.6:0/3982379397 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f12f619c700 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.853+0000 7f12f519a700 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 -- 172.21.15.6:0/3982379397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 msgr2=0x55da431be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 -- 172.21.15.6:0/3982379397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 msgr2=0x55da431b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 -- 172.21.15.6:0/3982379397 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da431ae6f0 con 0x55da430136b0 2023-11-17T00:11:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12f599b700 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0055b0 tx=0x7f12ec00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72ed7324bac89b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12e4bff700 1 -- 172.21.15.6:0/3982379397 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00ba30 con 0x55da430136b0 2023-11-17T00:11:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12e4bff700 1 -- 172.21.15.6:0/3982379397 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ec00f770 con 0x55da430136b0 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.854+0000 7f12e4bff700 1 -- 172.21.15.6:0/3982379397 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec003c00 con 0x55da430136b0 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- 172.21.15.6:0/3982379397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 msgr2=0x55da431c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0055b0 tx=0x7f12ec00fa00 comp rx=0 tx=0).stop 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- 172.21.15.6:0/3982379397 shutdown_connections 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431c5090 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 --2- 172.21.15.6:0/3982379397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- 172.21.15.6:0/3982379397 >> 172.21.15.6:0/3982379397 conn(0x55da431acf50 msgr2=0x55da431c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- 172.21.15.6:0/3982379397 shutdown_connections 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- 172.21.15.6:0/3982379397 wait complete. 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 -- start start 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.855+0000 7f130b492b80 1 Processor -- start 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da431c9770 con 0x55da430136b0 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43227de0 con 0x55da431c1b20 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da43227f20 con 0x55da431c1f90 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f130b492b80 10 client.0 _ll_register_callbacks cb 0x7f12e0009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43066/0 (socket says 172.21.15.6:43066) 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 -- 172.21.15.6:0/704813944 learned_addr learned my addr 172.21.15.6:0/704813944 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f519a700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f619c700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 -- 172.21.15.6:0/704813944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 msgr2=0x55da431c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 -- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 msgr2=0x55da431c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.856+0000 7f12f599b700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12f599b700 1 -- 172.21.15.6:0/704813944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da430cf130 con 0x55da430136b0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12f519a700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12f599b700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00ba60 tx=0x7f12ec00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca121021a78e5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec02f890 con 0x55da430136b0 2023-11-17T00:11:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f12ec02f9f0 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.857+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12ec01d040 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12ec003800 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 10 client.16002 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55da42f36d80 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12ec01d040 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12ec038a60 con 0x55da430136b0 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 20 client.16002 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da432290b0 0x55da4322b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 4 client.16002 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:11:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da430d7600 con 0x55da432290b0 2023-11-17T00:11:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.858+0000 7f12f519a700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da432290b0 0x55da4322b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.859+0000 7f12f519a700 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da432290b0 0x55da4322b490 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bc1a0029ae2e01 server_cookie=62017199918e0fa5 in_seq=0 out_seq=0 2023-11-17T00:11:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.859+0000 7f12d97fa700 10 client.16002 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.883+0000 7f12d97fa700 1 -- 172.21.15.6:0/704813944 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da430d7600 con 0x55da432290b0 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.883+0000 7f12d97fa700 10 client.16002 handle_command_reply: tid=0 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.883+0000 7f130b492b80 1 client.16002 shutdown 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.883+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 msgr2=0x55da431cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.883+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00ba60 tx=0x7f12ec00ba90 comp rx=0 tx=0).stop 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 shutdown_connections 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55da432290b0 0x55da4322b490 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55da430136b0 0x55da431cb2b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f12d97fa700 0 client.16002 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55da431c1f90 0x55da431c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 --2- 172.21.15.6:0/704813944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55da431c1b20 0x55da431c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 >> 172.21.15.6:0/704813944 conn(0x55da431acf50 msgr2=0x55da431b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 shutdown_connections 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 1 -- 172.21.15.6:0/704813944 wait complete. 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.884+0000 7f130b492b80 20 client.16002 trim_cache size 0 max 16384 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 msgr2=0x7f12dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 0x7f12dc0a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005e90 tx=0x7f12ec019040 comp rx=0 tx=0).stop 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 msgr2=0x7f12f019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ef30 tx=0x7f12e800c3a0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 shutdown_connections 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f12dc0a12c0 0x7f12dc0a3770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12f010a800 0x7f12f019ad70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f01012e0 0x7f12f01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 --2- 172.21.15.6:0/368952433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12f0101bf0 0x7f12f0196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 >> 172.21.15.6:0/368952433 conn(0x7f12f00763a0 msgr2=0x7f12f0102260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 shutdown_connections 2023-11-17T00:11:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:07.885+0000 7f12f7bff700 1 -- 172.21.15.6:0/368952433 wait complete. 2023-11-17T00:11:07.950 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:07.950 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:08.000 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:08.000 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:08.001 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 98 -lt 100 ']' 2023-11-17T00:11:08.001 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 98 + 1 2023-11-17T00:11:08.045 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=99 2023-11-17T00:11:08.046 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:08.105 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:08.105 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:08.108 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:08.108 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:08.109 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:08.723 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:08.723 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:08.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:08 smithi087 ceph-mon[131182]: pgmap v70: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-17T00:11:08.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:08 smithi186 ceph-mon[131487]: pgmap v70: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-17T00:11:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:08 smithi006 ceph-mon[97840]: pgmap v70: 129 pgs: 129 active+clean; 3.0 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-17T00:11:09.012 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:09.015 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:09.019 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:09.019 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:09.019 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 99 -lt 100 ']' 2023-11-17T00:11:09.020 INFO:tasks.workunit.client.0.smithi006.stderr:++ expr 99 + 1 2023-11-17T00:11:09.079 INFO:tasks.workunit.client.0.smithi006.stderr:+ c=100 2023-11-17T00:11:09.079 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:09.166 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:09.166 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:09.168 INFO:tasks.workunit.client.0.smithi006.stderr:+ c1=2 2023-11-17T00:11:09.168 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 3 2023-11-17T00:11:09.169 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-17T00:11:09.831 INFO:tasks.workunit.client.0.smithi006.stderr:++ getfacl d1 2023-11-17T00:11:09.831 INFO:tasks.workunit.client.0.smithi006.stderr:++ grep -c nobody:rw 2023-11-17T00:11:10.106 INFO:tasks.workunit.client.0.smithi006.stderr:+ c2=2 2023-11-17T00:11:10.106 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:10.217 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:10.219 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 2 -ne 2 ']' 2023-11-17T00:11:10.219 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 100 -lt 100 ']' 2023-11-17T00:11:10.282 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir d1 2023-11-17T00:11:10.336 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-17T00:11:10.365 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-17T00:11:10.368 INFO:tasks.workunit.client.0.smithi006.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-17T00:11:10.368 INFO:tasks.workunit.client.0.smithi006.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 -- 172.21.15.6:0/2434854818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 msgr2=0x7f16ac10c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 --2- 172.21.15.6:0/2434854818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac10c780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f169c009990 tx=0x7f169c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 -- 172.21.15.6:0/2434854818 shutdown_connections 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 --2- 172.21.15.6:0/2434854818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 0x7f16ac10f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 --2- 172.21.15.6:0/2434854818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac0ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 --2- 172.21.15.6:0/2434854818 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac10c780 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 -- 172.21.15.6:0/2434854818 >> 172.21.15.6:0/2434854818 conn(0x7f16ac0760d0 msgr2=0x7f16ac0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 -- 172.21.15.6:0/2434854818 shutdown_connections 2023-11-17T00:11:10.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.367+0000 7f16b3bff700 1 -- 172.21.15.6:0/2434854818 wait complete. 2023-11-17T00:11:10.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.368+0000 7f16b3bff700 1 Processor -- start 2023-11-17T00:11:10.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 -- start start 2023-11-17T00:11:10.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 0x7f16ac19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac19d590 con 0x7f16ac0ffa20 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac19d700 con 0x7f16ac10ccc0 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b3bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac19d8c0 con 0x7f16ac0ff110 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b119a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b119a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48930/0 (socket says 172.21.15.6:48930) 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b119a700 1 -- 172.21.15.6:0/4265338522 learned_addr learned my addr 172.21.15.6:0/4265338522 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b199b700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.369+0000 7f16b219c700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 0x7f16ac19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 -- 172.21.15.6:0/4265338522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 msgr2=0x7f16ac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 -- 172.21.15.6:0/4265338522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 msgr2=0x7f16ac19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 0x7f16ac19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 -- 172.21.15.6:0/4265338522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0095f0 con 0x7f16ac0ffa20 2023-11-17T00:11:10.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b199b700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:10.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b119a700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f169c0095c0 tx=0x7f169c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.374 INFO:tasks.workunit.client.0.smithi006.stderr:d1: system.posix_acl_access: No such attribute 2023-11-17T00:11:10.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16a2ffd700 1 -- 172.21.15.6:0/4265338522 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c01d070 con 0x7f16ac0ffa20 2023-11-17T00:11:10.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.370+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac1125b0 con 0x7f16ac0ffa20 2023-11-17T00:11:10.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.371+0000 7f16a2ffd700 1 -- 172.21.15.6:0/4265338522 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169c005050 con 0x7f16ac0ffa20 2023-11-17T00:11:10.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.371+0000 7f16a2ffd700 1 -- 172.21.15.6:0/4265338522 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c004b30 con 0x7f16ac0ffa20 2023-11-17T00:11:10.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.371+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac112420 con 0x7f16ac0ffa20 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:+ return 0 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:+ getfattr -n system.posix_acl_default . 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:.: system.posix_acl_default: No such attribute 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:+ return 0 2023-11-17T00:11:10.376 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir d1 2023-11-17T00:11:10.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.374+0000 7f16a2ffd700 1 -- 172.21.15.6:0/4265338522 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f169c004cb0 con 0x7f16ac0ffa20 2023-11-17T00:11:10.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.376+0000 7f16a2ffd700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 0x7f16980a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.376+0000 7f16a2ffd700 1 -- 172.21.15.6:0/4265338522 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f169c0cdf10 con 0x7f16ac0ffa20 2023-11-17T00:11:10.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.376+0000 7f16b199b700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 0x7f16980a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.377+0000 7f16b199b700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 0x7f16980a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005950 tx=0x7f16a80058e0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd .. 2023-11-17T00:11:10.379 INFO:tasks.workunit.client.0.smithi006.stderr:+ rmdir testdir 2023-11-17T00:11:10.394 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo OK 2023-11-17T00:11:10.395 INFO:tasks.workunit.client.0.smithi006.stdout:OK 2023-11-17T00:11:10.396 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:11:10.396 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:11:10.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.735+0000 7f16c75d2b80 1 Processor -- start 2023-11-17T00:11:10.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 -- start start 2023-11-17T00:11:10.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a7cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a81d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be539820e0 con 0x55be53a7d4f0 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be539d4920 con 0x55be53a7cbe0 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be539cddb0 con 0x55be53a81930 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16b119a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16b119a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48934/0 (socket says 172.21.15.6:48934) 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.736+0000 7f16b119a700 1 -- 172.21.15.6:0/2756640121 learned_addr learned my addr 172.21.15.6:0/2756640121 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b219c700 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a81d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b199b700 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a7cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 -- 172.21.15.6:0/2756640121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 msgr2=0x55be53a81d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a81d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 -- 172.21.15.6:0/2756640121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 msgr2=0x55be53a7cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a7cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 -- 172.21.15.6:0/2756640121 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5384f6b0 con 0x55be53a7d4f0 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f16b119a700 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f169c097730 tx=0x7f169c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5501d7a684202ad server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f1697fff700 1 -- 172.21.15.6:0/2756640121 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0a3080 con 0x55be53a7d4f0 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f1697fff700 1 -- 172.21.15.6:0/2756640121 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169c099060 con 0x55be53a7d4f0 2023-11-17T00:11:10.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.737+0000 7f1697fff700 1 -- 172.21.15.6:0/2756640121 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c09a410 con 0x55be53a7d4f0 2023-11-17T00:11:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 -- 172.21.15.6:0/2756640121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 msgr2=0x55be53a813f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f169c097730 tx=0x7f169c0057d0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 -- 172.21.15.6:0/2756640121 shutdown_connections 2023-11-17T00:11:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a81d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a7cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/2756640121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a813f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 -- 172.21.15.6:0/2756640121 >> 172.21.15.6:0/2756640121 conn(0x55be53a723e0 msgr2=0x55be53a77bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 -- 172.21.15.6:0/2756640121 shutdown_connections 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.738+0000 7f16c75d2b80 1 -- 172.21.15.6:0/2756640121 wait complete. 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.739+0000 7f16c75d2b80 1 -- start start 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.739+0000 7f16c75d2b80 1 Processor -- start 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a86930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5396f960 con 0x55be53a7d4f0 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5396f600 con 0x55be53a7cbe0 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be538519a0 con 0x55be53a81930 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16c75d2b80 10 client.0 _ll_register_callbacks cb 0x55be5398d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:10.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b119a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40744/0 (socket says 172.21.15.6:40744) 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 -- 172.21.15.6:0/1305915869 learned_addr learned my addr 172.21.15.6:0/1305915869 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b219c700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a86930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 -- 172.21.15.6:0/1305915869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 msgr2=0x55be53a86930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a86930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 -- 172.21.15.6:0/1305915869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 msgr2=0x55be53a86320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a86320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.740+0000 7f16b199b700 1 -- 172.21.15.6:0/1305915869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac112420 con 0x55be53a7cbe0 2023-11-17T00:11:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16b199b700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f16a80042e0 tx=0x7f16a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=197ceee24c5b50c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16957fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8017040 con 0x55be53a7cbe0 2023-11-17T00:11:10.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be53a6fba0 con 0x55be53a7cbe0 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f16a8003a90 con 0x55be53a7cbe0 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a8003c90 con 0x55be53a7cbe0 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16c75d2b80 10 client.26075 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.741+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55be53994210 con 0x55be53a7cbe0 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be53a6fba0 con 0x55be53a7cbe0 2023-11-17T00:11:10.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16a80204a0 con 0x55be53a7cbe0 2023-11-17T00:11:10.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:10.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:10.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 20 client.26075 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:10.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53a965f0 0x55be53a98a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 4 client.26075 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be53990120 con 0x55be53a965f0 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.742+0000 7f16b119a700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53a965f0 0x55be53a98a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.743+0000 7f16b119a700 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53a965f0 0x55be53a98a00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8d4e1ece04c6 server_cookie=8c1fd4d5cb1f217d in_seq=0 out_seq=0 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.743+0000 7f16957fa700 10 client.26075 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.743+0000 7f16957fa700 1 -- 172.21.15.6:0/1305915869 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be53990120 con 0x55be53a965f0 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.743+0000 7f16957fa700 10 client.26075 handle_command_reply: tid=0 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 client.26075 shutdown 2023-11-17T00:11:10.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 msgr2=0x55be53a890a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f16a80042e0 tx=0x7f16a8004750 comp rx=0 tx=0).stop 2023-11-17T00:11:10.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 shutdown_connections 2023-11-17T00:11:10.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a81930 0x55be53a86930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7cbe0 0x55be53a890a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53a965f0 0x55be53a98a00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/1305915869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16957fa700 0 client.26075 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 >> 172.21.15.6:0/1305915869 conn(0x55be53a723e0 msgr2=0x55be53a76e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 shutdown_connections 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 1 -- 172.21.15.6:0/1305915869 wait complete. 2023-11-17T00:11:10.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.744+0000 7f16c75d2b80 20 client.26075 trim_cache size 0 max 16384 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.763+0000 7f16c75d2b80 1 Processor -- start 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.763+0000 7f16c75d2b80 1 -- start start 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a83d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a81930 0x55be53a80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a80260 con 0x55be53a7d4f0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a79240 con 0x55be53a81930 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a84880 con 0x55be53a7cbe0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:46206/0 (socket says 172.21.15.6:46206) 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 -- 172.21.15.6:0/380802962 learned_addr learned my addr 172.21.15.6:0/380802962 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b119a700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a83d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b219c700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a81930 0x55be53a80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 -- 172.21.15.6:0/380802962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a81930 msgr2=0x55be53a80f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a81930 0x55be53a80f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 -- 172.21.15.6:0/380802962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 msgr2=0x55be53a83d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a83d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b199b700 1 -- 172.21.15.6:0/380802962 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be53993f60 con 0x55be53a7cbe0 2023-11-17T00:11:10.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.764+0000 7f16b119a700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a83d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.765+0000 7f16b199b700 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004610 tx=0x7f16a800f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7feea10197605c0d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.765+0000 7f16a0bff700 1 -- 172.21.15.6:0/380802962 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8020610 con 0x55be53a7cbe0 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.765+0000 7f16a0bff700 1 -- 172.21.15.6:0/380802962 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a8020aa0 con 0x55be53a7cbe0 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.765+0000 7f16a0bff700 1 -- 172.21.15.6:0/380802962 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80299e0 con 0x55be53a7cbe0 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- 172.21.15.6:0/380802962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 msgr2=0x55be53a8d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004610 tx=0x7f16a800f9d0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- 172.21.15.6:0/380802962 shutdown_connections 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a81930 0x55be53a80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/380802962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a7d4f0 0x55be53a83d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- 172.21.15.6:0/380802962 >> 172.21.15.6:0/380802962 conn(0x55be53a723e0 msgr2=0x55be53a80860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- 172.21.15.6:0/380802962 shutdown_connections 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- 172.21.15.6:0/380802962 wait complete. 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.766+0000 7f16c75d2b80 1 -- start start 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 Processor -- start 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a81930 0x55be53a8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a8c2f0 con 0x55be53a81930 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a8c430 con 0x55be53a7d4f0 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53a8c5f0 con 0x55be53a7cbe0 2023-11-17T00:11:10.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16c75d2b80 10 client.0 _ll_register_callbacks cb 0x7f16a800fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:10.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16b199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16b119a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.767+0000 7f16b219c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a81930 0x55be53a8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:46222/0 (socket says 172.21.15.6:46222) 2023-11-17T00:11:10.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40762/0 (socket says 172.21.15.6:40762) 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b199b700 1 -- 172.21.15.6:0/4282909393 learned_addr learned my addr 172.21.15.6:0/4282909393 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 -- 172.21.15.6:0/4282909393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 msgr2=0x55be53a8d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 -- 172.21.15.6:0/4282909393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a81930 msgr2=0x55be53a8bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a81930 0x55be53a8bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 -- 172.21.15.6:0/4282909393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be53994210 con 0x55be53a7d4f0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16b119a700 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f169c09ac50 tx=0x7f169c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5471acea94595880 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16957fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c09aa90 con 0x55be53a7d4f0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f169c0b3240 con 0x55be53a7d4f0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.768+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f169c099060 con 0x55be53a7d4f0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f169c0b3400 con 0x55be53a7d4f0 2023-11-17T00:11:10.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 10 client.26081 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55be537e4ce0 con 0x55be53a7d4f0 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169c099060 con 0x55be53a7d4f0 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f169c09af20 con 0x55be53a7d4f0 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 20 client.26081 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53aec650 0x55be53aeea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 4 client.26081 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.769+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55be5399c6e0 con 0x55be53aec650 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.770+0000 7f16b199b700 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53aec650 0x55be53aeea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.770+0000 7f16b199b700 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53aec650 0x55be53aeea30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a9ad948ae07a88 server_cookie=dafd2a4859d4c393 in_seq=0 out_seq=0 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.770+0000 7f16957fa700 10 client.26081 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:10.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16957fa700 1 -- 172.21.15.6:0/4282909393 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55be5399c6e0 con 0x55be53aec650 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16957fa700 10 client.26081 handle_command_reply: tid=0 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 client.26081 shutdown 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 msgr2=0x55be53a8b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f169c09ac50 tx=0x7f169c009310 comp rx=0 tx=0).stop 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 shutdown_connections 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55be53aec650 0x55be53aeea30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55be53a81930 0x55be53a8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16957fa700 0 client.26081 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be53a7cbe0 0x55be53a8d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 --2- 172.21.15.6:0/4282909393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be53a7d4f0 0x55be53a8b820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 >> 172.21.15.6:0/4282909393 conn(0x55be53a723e0 msgr2=0x55be53a7c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.771+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 shutdown_connections 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16c75d2b80 1 -- 172.21.15.6:0/4282909393 wait complete. 2023-11-17T00:11:10.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16c75d2b80 20 client.26081 trim_cache size 0 max 16384 2023-11-17T00:11:10.793 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:11:10.793 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:11:10.793 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "b18298a9-25a7-4f3d-a1bb-186552523790", 2023-11-17T00:11:10.793 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:11:10.793 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:11:10.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 msgr2=0x7f16980a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 0x7f16980a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005950 tx=0x7f16a80058e0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 msgr2=0x7f16ac198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:10.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f169c0095c0 tx=0x7f169c00f690 comp rx=0 tx=0).stop 2023-11-17T00:11:10.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 shutdown_connections 2023-11-17T00:11:10.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f16980a1340 0x7f16980a37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ac10ccc0 0x7f16ac19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16ac0ff110 0x7f16ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 --2- 172.21.15.6:0/4265338522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f16ac0ffa20 0x7f16ac198af0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:10.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.772+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 >> 172.21.15.6:0/4265338522 conn(0x7f16ac0760d0 msgr2=0x7f16ac0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.773+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 shutdown_connections 2023-11-17T00:11:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:10.773+0000 7f16b3bff700 1 -- 172.21.15.6:0/4265338522 wait complete. 2023-11-17T00:11:10.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:10 smithi006 ceph-mon[97840]: pgmap v71: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 59 KiB/s wr, 25 op/s 2023-11-17T00:11:10.831 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:10 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:11:10.833 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:11:10.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:10 smithi087 ceph-mon[131182]: pgmap v71: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 59 KiB/s wr, 25 op/s 2023-11-17T00:11:10.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:10 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:11:10.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:10 smithi186 ceph-mon[131487]: pgmap v71: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 59 KiB/s wr, 25 op/s 2023-11-17T00:11:10.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:10 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:11:11.282 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-17T00:11:11.283 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-17T00:11:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2023-11-17T00:11:11.369 INFO:tasks.workunit.client.0.smithi006.stderr:+ file=test_chmod.154596 2023-11-17T00:11:11.369 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo foo 2023-11-17T00:11:11.375 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.375 INFO:tasks.workunit.client.0.smithi006.stderr:+ chmod 400 test_chmod.154596 2023-11-17T00:11:11.388 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.388 INFO:tasks.workunit.client.0.smithi006.stderr:+ check_perms test_chmod.154596 -r-------- 2023-11-17T00:11:11.388 INFO:tasks.workunit.client.0.smithi006.stderr:+ file=test_chmod.154596 2023-11-17T00:11:11.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ ls -la test_chmod.154596 2023-11-17T00:11:11.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 17 00:11 test_chmod.154596' 2023-11-17T00:11:11.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ perms=-r-------- 2023-11-17T00:11:11.410 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 17 00:11 test_chmod.154596 2023-11-17T00:11:11.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2023-11-17T00:11:11.505 INFO:tasks.workunit.client.0.smithi006.stderr:+ test -r-------- '!=' -r--------. 2023-11-17T00:11:11.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 17 00:11 test_chmod.154596 2023-11-17T00:11:11.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2023-11-17T00:11:11.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ test -r--------. '!=' -r--------. 2023-11-17T00:11:11.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ set +e 2023-11-17T00:11:11.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo bar 2023-11-17T00:11:11.518 INFO:tasks.workunit.client.0.smithi006.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.154596: Permission denied 2023-11-17T00:11:11.519 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 1 = 0 2023-11-17T00:11:11.519 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2023-11-17T00:11:11.519 INFO:tasks.workunit.client.0.smithi006.stderr:+ chmod 600 test_chmod.154596 2023-11-17T00:11:11.520 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo bar 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ check_perms test_chmod.154596 -rw------- 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ file=test_chmod.154596 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ ls -la test_chmod.154596 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 17 00:11 test_chmod.154596' 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:+ perms=-rw------- 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 17 00:11 test_chmod.154596 2023-11-17T00:11:11.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2023-11-17T00:11:11.523 INFO:tasks.workunit.client.0.smithi006.stderr:+ test -rw------- '!=' -rw-------. 2023-11-17T00:11:11.524 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 17 00:11 test_chmod.154596 2023-11-17T00:11:11.524 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2023-11-17T00:11:11.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-17T00:11:11.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo foo 2023-11-17T00:11:11.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ test 0 '!=' 0 2023-11-17T00:11:11.529 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:11:11.529 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:11:11.616 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-17T00:11:11.617 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-17T00:11:11.679 INFO:tasks.workunit.client.0.smithi006.stderr:+ set -e 2023-11-17T00:11:11.679 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir -p testdir 2023-11-17T00:11:11.683 INFO:tasks.workunit.client.0.smithi006.stderr:+ file=test_chmod.154656 2023-11-17T00:11:11.683 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo foo 2023-11-17T00:11:11.684 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo chmod 600 testdir 2023-11-17T00:11:11.719 INFO:tasks.workunit.client.0.smithi006.stderr:+ expect_failure cat testdir/test_chmod.154656 2023-11-17T00:11:11.719 INFO:tasks.workunit.client.0.smithi006.stderr:+ cat testdir/test_chmod.154656 2023-11-17T00:11:11.745 INFO:tasks.workunit.client.0.smithi006.stderr:cat: testdir/test_chmod.154656: Permission denied 2023-11-17T00:11:11.745 INFO:tasks.workunit.client.0.smithi006.stderr:+ return 0 2023-11-17T00:11:11.745 INFO:tasks.workunit.client.0.smithi006.stderr:+ sudo cat testdir/test_chmod.154656 2023-11-17T00:11:11.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:11 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:11:11.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:11 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:11 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:11:11.766 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:11 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.775 INFO:tasks.workunit.client.0.smithi006.stdout:foo 2023-11-17T00:11:11.776 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:11:11.776 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:11:11.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:11 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:11:11.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:11 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:11 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:11:11.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:11 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.864 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-17T00:11:11.865 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-17T00:11:11.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:11 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:11:11.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:11 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:11 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:11:11.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:11 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1012400399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8240746a0 msgr2=0x7fc824072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1012400399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8240746a0 0x7fc824072d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc814009990 tx=0x7fc814009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1012400399 shutdown_connections 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1012400399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 0x7fc82410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1012400399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8240746a0 0x7fc824072d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1012400399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc824074b00 0x7fc824073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1012400399 >> 172.21.15.6:0/1012400399 conn(0x7fc824077500 msgr2=0x7fc824077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1012400399 shutdown_connections 2023-11-17T00:11:11.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.892+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1012400399 wait complete. 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.893+0000 7fc82a1ff700 1 Processor -- start 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.893+0000 7fc82a1ff700 1 -- start start 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc824074b00 0x7fc824198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 0x7fc82419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82419d580 con 0x7fc8240746a0 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82419d6f0 con 0x7fc82410ce50 2023-11-17T00:11:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc82a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82419d8b0 con 0x7fc824074b00 2023-11-17T00:11:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48974/0 (socket says 172.21.15.6:48974) 2023-11-17T00:11:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 -- 172.21.15.6:0/1881898538 learned_addr learned my addr 172.21.15.6:0/1881898538 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc822ffd700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc824074b00 0x7fc824198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc823fff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 0x7fc82419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 -- 172.21.15.6:0/1881898538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc824074b00 msgr2=0x7fc824198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc824074b00 0x7fc824198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 -- 172.21.15.6:0/1881898538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 msgr2=0x7fc82419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 0x7fc82419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.894+0000 7fc8237fe700 1 -- 172.21.15.6:0/1881898538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009520 con 0x7fc8240746a0 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc814005910 tx=0x7fc814005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc820ff9700 1 -- 172.21.15.6:0/1881898538 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81401d070 con 0x7fc8240746a0 2023-11-17T00:11:11.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8140095f0 con 0x7fc8240746a0 2023-11-17T00:11:11.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc820ff9700 1 -- 172.21.15.6:0/1881898538 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc814022470 con 0x7fc8240746a0 2023-11-17T00:11:11.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc820ff9700 1 -- 172.21.15.6:0/1881898538 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814017650 con 0x7fc8240746a0 2023-11-17T00:11:11.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.895+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824112550 con 0x7fc8240746a0 2023-11-17T00:11:11.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.898+0000 7fc820ff9700 1 -- 172.21.15.6:0/1881898538 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc8140225e0 con 0x7fc8240746a0 2023-11-17T00:11:11.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.900+0000 7fc820ff9700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 0x7fc80c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.900+0000 7fc820ff9700 1 -- 172.21.15.6:0/1881898538 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc8140ce3d0 con 0x7fc8240746a0 2023-11-17T00:11:11.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.901+0000 7fc822ffd700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 0x7fc80c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.901+0000 7fc83db7eb80 1 Processor -- start 2023-11-17T00:11:11.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.901+0000 7fc83db7eb80 1 -- start start 2023-11-17T00:11:11.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.901+0000 7fc822ffd700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 0x7fc80c0a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc824199bb0 tx=0x7fc818009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 0x55adef6f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef5f3060 con 0x55adef6edb60 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6458a0 con 0x55adef6ee470 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef63ed30 con 0x55adef6f28b0 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48982/0 (socket says 172.21.15.6:48982) 2023-11-17T00:11:11.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 -- 172.21.15.6:0/1546870080 learned_addr learned my addr 172.21.15.6:0/1546870080 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc822ffd700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc823fff700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 0x55adef6f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 -- 172.21.15.6:0/1546870080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 msgr2=0x55adef6f2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 0x55adef6f2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 -- 172.21.15.6:0/1546870080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 msgr2=0x55adef6f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc8237fe700 1 -- 172.21.15.6:0/1546870080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adef4c0630 con 0x55adef6edb60 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.902+0000 7fc823fff700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 0x55adef6f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.903+0000 7fc8237fe700 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a3010 tx=0x7fc814009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f26b55e7000be3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.903+0000 7fc809ffb700 1 -- 172.21.15.6:0/1546870080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81409aae0 con 0x55adef6edb60 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.903+0000 7fc809ffb700 1 -- 172.21.15.6:0/1546870080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc81409b180 con 0x55adef6edb60 2023-11-17T00:11:11.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.903+0000 7fc809ffb700 1 -- 172.21.15.6:0/1546870080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81409b640 con 0x55adef6edb60 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 -- 172.21.15.6:0/1546870080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 msgr2=0x55adef6edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a3010 tx=0x7fc814009310 comp rx=0 tx=0).stop 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 -- 172.21.15.6:0/1546870080 shutdown_connections 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6edb60 0x55adef6edf30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6f28b0 0x55adef6f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/1546870080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 -- 172.21.15.6:0/1546870080 >> 172.21.15.6:0/1546870080 conn(0x55adef6e3360 msgr2=0x55adef6e8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 -- 172.21.15.6:0/1546870080 shutdown_connections 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.904+0000 7fc83db7eb80 1 -- 172.21.15.6:0/1546870080 wait complete. 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.905+0000 7fc83db7eb80 1 -- start start 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.905+0000 7fc83db7eb80 1 Processor -- start 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.905+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 0x55adef6fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.905+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef5e08e0 con 0x55adef6f28b0 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef5e0580 con 0x55adef6edb60 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef4c2920 con 0x55adef6ee470 2023-11-17T00:11:11.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc83db7eb80 10 client.0 _ll_register_callbacks cb 0x55adef5fe780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc823fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:46238/0 (socket says 172.21.15.6:46238) 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 -- 172.21.15.6:0/3610019184 learned_addr learned my addr 172.21.15.6:0/3610019184 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 0x55adef6fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 -- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 msgr2=0x55adef6fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 0x55adef6fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 -- 172.21.15.6:0/3610019184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 msgr2=0x55adef6f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc822ffd700 1 -- 172.21.15.6:0/3610019184 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824112550 con 0x55adef6ee470 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.906+0000 7fc8237fe700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 0x55adef6fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:11.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc81800edd0 tx=0x7fc8180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5dc96e9d300d9e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818022070 con 0x55adef6ee470 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adef6e0b20 con 0x55adef6ee470 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc81800ee00 con 0x55adef6ee470 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc81801dd10 con 0x55adef6ee470 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc83db7eb80 10 client.26074 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:11.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.907+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55adef605190 con 0x55adef6ee470 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adef6e0b20 con 0x55adef6ee470 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc818003c80 con 0x55adef6ee470 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 10 client.26074 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 10 client.26074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 20 client.26074 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:11.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef707560 0x55adef709970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 4 client.26074 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adef6010a0 con 0x55adef707560 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.908+0000 7fc8237fe700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef707560 0x55adef709970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.909+0000 7fc8237fe700 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef707560 0x55adef709970 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9744709fbfa687e server_cookie=6be38b139ff2e50c in_seq=0 out_seq=0 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.909+0000 7fc802ffd700 10 client.26074 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.909+0000 7fc802ffd700 1 -- 172.21.15.6:0/3610019184 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55adef6010a0 con 0x55adef707560 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.909+0000 7fc802ffd700 10 client.26074 handle_command_reply: tid=0 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 client.26074 shutdown 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 msgr2=0x55adef6f7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc81800edd0 tx=0x7fc8180047c0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 shutdown_connections 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef707560 0x55adef709970 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6edb60 0x55adef6fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3610019184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6ee470 0x55adef6f7220 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 >> 172.21.15.6:0/3610019184 conn(0x55adef6e3360 msgr2=0x55adef6f3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc802ffd700 0 client.26074 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 shutdown_connections 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3610019184 wait complete. 2023-11-17T00:11:11.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.910+0000 7fc83db7eb80 20 client.26074 trim_cache size 0 max 16384 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.929+0000 7fc83db7eb80 1 Processor -- start 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.929+0000 7fc83db7eb80 1 -- start start 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6f28b0 0x55adef6f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6f8390 con 0x55adef6ee470 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6010a0 con 0x55adef6f28b0 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6e9710 con 0x55adef6edb60 2023-11-17T00:11:11.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc822ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc822ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49006/0 (socket says 172.21.15.6:49006) 2023-11-17T00:11:11.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc822ffd700 1 -- 172.21.15.6:0/3509309667 learned_addr learned my addr 172.21.15.6:0/3509309667 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:11.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc823fff700 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6f28b0 0x55adef6f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc8237fe700 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.930+0000 7fc822ffd700 1 -- 172.21.15.6:0/3509309667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 msgr2=0x55adef6e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc822ffd700 1 -- 172.21.15.6:0/3509309667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6f28b0 msgr2=0x55adef6f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6f28b0 0x55adef6f9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc822ffd700 1 -- 172.21.15.6:0/3509309667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adef6e7820 con 0x55adef6ee470 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc822ffd700 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc818016040 tx=0x7fc818011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc00f6177677e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc80a7fc700 1 -- 172.21.15.6:0/3509309667 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818039070 con 0x55adef6ee470 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc80a7fc700 1 -- 172.21.15.6:0/3509309667 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc818026650 con 0x55adef6ee470 2023-11-17T00:11:11.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.931+0000 7fc80a7fc700 1 -- 172.21.15.6:0/3509309667 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818022070 con 0x55adef6ee470 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3509309667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 msgr2=0x55adef6fe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc818016040 tx=0x7fc818011ab0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3509309667 shutdown_connections 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6f28b0 0x55adef6f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/3509309667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6ee470 0x55adef6fe7b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3509309667 >> 172.21.15.6:0/3509309667 conn(0x55adef6e3360 msgr2=0x55adef702f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3509309667 shutdown_connections 2023-11-17T00:11:11.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.932+0000 7fc83db7eb80 1 -- 172.21.15.6:0/3509309667 wait complete. 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 -- start start 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 Processor -- start 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6f6700 con 0x55adef6f28b0 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6f6840 con 0x55adef6ee470 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adef6f6a00 con 0x55adef6edb60 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.933+0000 7fc83db7eb80 10 client.0 _ll_register_callbacks cb 0x7fc81800e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc823fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:46256/0 (socket says 172.21.15.6:46256) 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc8237fe700 1 -- 172.21.15.6:0/604731253 learned_addr learned my addr 172.21.15.6:0/604731253 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 -- 172.21.15.6:0/604731253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 msgr2=0x55adef6fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 -- 172.21.15.6:0/604731253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 msgr2=0x55adef6f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 -- 172.21.15.6:0/604731253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adef605190 con 0x55adef6ee470 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc822ffd700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f59f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8180265d0 tx=0x7fc81801da40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b01b3ffad9a655b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.934+0000 7fc802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818022050 con 0x55adef6ee470 2023-11-17T00:11:11.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc81409b180 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc818005110 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc818039040 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 10 client.26099 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55adef46cde0 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc81409b180 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc818035430 con 0x55adef6ee470 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 10 client.26099 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 20 client.26099 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef75ba30 0x55adef75de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 4 client.26099 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:11:11.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.935+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55adef60d660 con 0x55adef75ba30 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc8237fe700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef75ba30 0x55adef75de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc8237fe700 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef75ba30 0x55adef75de40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c69a3a54575029 server_cookie=539326f5f57a816f in_seq=0 out_seq=0 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc802ffd700 10 client.26099 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc802ffd700 1 -- 172.21.15.6:0/604731253 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55adef60d660 con 0x55adef75ba30 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc802ffd700 10 client.26099 handle_command_reply: tid=0 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc83db7eb80 1 client.26099 shutdown 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.936+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 msgr2=0x55adef6f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f59f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc8180265d0 tx=0x7fc81801da40 comp rx=0 tx=0).stop 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 shutdown_connections 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55adef75ba30 0x55adef75de40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55adef6f28b0 0x55adef6f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55adef6edb60 0x55adef6fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 --2- 172.21.15.6:0/604731253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55adef6ee470 0x55adef6f59f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc802ffd700 0 client.26099 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 >> 172.21.15.6:0/604731253 conn(0x55adef6e3360 msgr2=0x55adef6f08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 shutdown_connections 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 1 -- 172.21.15.6:0/604731253 wait complete. 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc83db7eb80 20 client.26099 trim_cache size 0 max 16384 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:11:11.946 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: "b18298a9-25a7-4f3d-a1bb-186552523790": { 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "b18298a9-25a7-4f3d-a1bb-186552523790", 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:11:11.947 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 msgr2=0x7fc80c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 0x7fc80c0a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc824199bb0 tx=0x7fc818009260 comp rx=0 tx=0).stop 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 msgr2=0x7fc8241985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.937+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc814005910 tx=0x7fc814005720 comp rx=0 tx=0).stop 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 shutdown_connections 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc80c0a12c0 0x7fc80c0a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8240746a0 0x7fc8241985a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc82410ce50 0x7fc82419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 --2- 172.21.15.6:0/1881898538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc824074b00 0x7fc824198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:11.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 >> 172.21.15.6:0/1881898538 conn(0x7fc824077500 msgr2=0x7fc824104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:11.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 shutdown_connections 2023-11-17T00:11:11.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:11.938+0000 7fc82a1ff700 1 -- 172.21.15.6:0/1881898538 wait complete. 2023-11-17T00:11:11.963 INFO:tasks.cephfs.filesystem:scrub status for tag:b18298a9-25a7-4f3d-a1bb-186552523790 - {'path': '/', 'tag': 'b18298a9-25a7-4f3d-a1bb-186552523790', 'options': 'recursive,force'} 2023-11-17T00:11:12.223 INFO:tasks.workunit.client.0.smithi006.stdout:ok 2023-11-17T00:11:12.230 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:11:12.230 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:11:12.315 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-17T00:11:12.315 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-17T00:11:12.419 INFO:tasks.workunit.client.0.smithi006.stdout:creating folder hierarchy 2023-11-17T00:11:12.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:12 smithi006 ceph-mon[97840]: pgmap v72: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:12.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:12 smithi087 ceph-mon[131182]: pgmap v72: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:12.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:12 smithi186 ceph-mon[131487]: pgmap v72: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: pgmap v73: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:11:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:14 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:11:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: pgmap v73: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:11:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:11:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:11:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:11:14.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:11:14.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:11:14.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:14 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: pgmap v73: 129 pgs: 129 active+clean; 3.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 22 op/s 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:11:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:14 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:11:15.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:15 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:11:15.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:15 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:11:15.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:15 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:11:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:16 smithi087 ceph-mon[131182]: pgmap v74: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 70 KiB/s wr, 26 op/s 2023-11-17T00:11:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:16 smithi186 ceph-mon[131487]: pgmap v74: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 70 KiB/s wr, 26 op/s 2023-11-17T00:11:16.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:16 smithi006 ceph-mon[97840]: pgmap v74: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 70 KiB/s wr, 26 op/s 2023-11-17T00:11:18.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:18 smithi087 ceph-mon[131182]: pgmap v75: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-11-17T00:11:18.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:18 smithi186 ceph-mon[131487]: pgmap v75: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-11-17T00:11:18.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:18 smithi006 ceph-mon[97840]: pgmap v75: 129 pgs: 129 active+clean; 3.3 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-11-17T00:11:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:20 smithi087 ceph-mon[131182]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 826 KiB/s wr, 25 op/s 2023-11-17T00:11:20.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:20 smithi186 ceph-mon[131487]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 826 KiB/s wr, 25 op/s 2023-11-17T00:11:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:20 smithi006 ceph-mon[97840]: pgmap v76: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 826 KiB/s wr, 25 op/s 2023-11-17T00:11:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 -- 172.21.15.6:0/3356392947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 msgr2=0x7f52a810b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/3356392947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a810b710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f52a000b110 tx=0x7f52a000aa40 comp rx=0 tx=0).stop 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 -- 172.21.15.6:0/3356392947 shutdown_connections 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/3356392947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a810b710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/3356392947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a81048e0 0x7f52a8104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/3356392947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a80fe920 0x7f52a80fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 -- 172.21.15.6:0/3356392947 >> 172.21.15.6:0/3356392947 conn(0x7f52a8074600 msgr2=0x7f52a8074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.137+0000 7f52ad1ff700 1 -- 172.21.15.6:0/3356392947 shutdown_connections 2023-11-17T00:11:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.138+0000 7f52ad1ff700 1 -- 172.21.15.6:0/3356392947 wait complete. 2023-11-17T00:11:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.139+0000 7f52ad1ff700 1 Processor -- start 2023-11-17T00:11:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.139+0000 7f52ad1ff700 1 -- start start 2023-11-17T00:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 0x7f52a8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a81048e0 0x7f52a819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a819d4e0 con 0x7f52a81048e0 2023-11-17T00:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a819d650 con 0x7f52a80ff2d0 2023-11-17T00:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52ad1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a819d810 con 0x7f52a80fe920 2023-11-17T00:11:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52a659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52a659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:36610/0 (socket says 172.21.15.6:36610) 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52a659c700 1 -- 172.21.15.6:0/233325804 learned_addr learned my addr 172.21.15.6:0/233325804 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 0x7f52a8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.140+0000 7f52a759e700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a81048e0 0x7f52a819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 -- 172.21.15.6:0/233325804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 msgr2=0x7f52a8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 -- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a81048e0 msgr2=0x7f52a819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a81048e0 0x7f52a819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 -- 172.21.15.6:0/233325804 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c00b730 con 0x7f52a80fe920 2023-11-17T00:11:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.141+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 0x7f52a8198500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f529800e9c0 tx=0x7f529800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.142+0000 7f528ffff700 1 -- 172.21.15.6:0/233325804 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298004220 con 0x7f52a80fe920 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.142+0000 7f528ffff700 1 -- 172.21.15.6:0/233325804 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5298004380 con 0x7f52a80fe920 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.142+0000 7f528ffff700 1 -- 172.21.15.6:0/233325804 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298003be0 con 0x7f52a80fe920 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.142+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a000ad20 con 0x7f52a80fe920 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.142+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a810e130 con 0x7f52a80fe920 2023-11-17T00:11:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.147+0000 7f52c0bb1b80 1 Processor -- start 2023-11-17T00:11:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.147+0000 7f52c0bb1b80 1 -- start start 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f528ffff700 1 -- 172.21.15.6:0/233325804 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5298003d40 con 0x7f52a80fe920 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410b946b0 0x55c410d40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410d41910 0x55c410d45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410c3e000 con 0x55c410d41910 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410c90840 con 0x55c410d414a0 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410c89cd0 con 0x55c410b946b0 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:36626/0 (socket says 172.21.15.6:36626) 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.148+0000 7f52a6d9d700 1 -- 172.21.15.6:0/1946793113 learned_addr learned my addr 172.21.15.6:0/1946793113 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a659c700 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410d41910 0x55c410d45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 -- 172.21.15.6:0/1946793113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410b946b0 msgr2=0x55c410d40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410b946b0 0x55c410d40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 -- 172.21.15.6:0/1946793113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410d41910 msgr2=0x55c410d45b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410d41910 0x55c410d45b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 -- 172.21.15.6:0/1946793113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c410b0b5d0 con 0x55c410d414a0 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52980ca6e0 tx=0x7f52980d5540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228fc4688f10e675 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.149+0000 7f528ffff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 0x7f52900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.150+0000 7f528ffff700 1 -- 172.21.15.6:0/233325804 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5298014070 con 0x7f52a80fe920 2023-11-17T00:11:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.150+0000 7f52a759e700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 0x7f52900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.150+0000 7f528d7fa700 1 -- 172.21.15.6:0/1946793113 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298096460 con 0x55c410d414a0 2023-11-17T00:11:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.150+0000 7f528d7fa700 1 -- 172.21.15.6:0/1946793113 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5298099ef0 con 0x55c410d414a0 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.150+0000 7f528d7fa700 1 -- 172.21.15.6:0/1946793113 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529801e2e0 con 0x55c410d414a0 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1946793113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 msgr2=0x55c410d455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52a759e700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 0x7f52900a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52a000b780 tx=0x7f52a0005040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52980ca6e0 tx=0x7f52980d5540 comp rx=0 tx=0).stop 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1946793113 shutdown_connections 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410d41910 0x55c410d45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410b946b0 0x55c410d40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1946793113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d455f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.151+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1946793113 >> 172.21.15.6:0/1946793113 conn(0x55c410d2df50 msgr2=0x55c410d33270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.152+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1946793113 shutdown_connections 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.152+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1946793113 wait complete. 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.152+0000 7f52c0bb1b80 1 -- start start 2023-11-17T00:11:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 Processor -- start 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410c2b880 con 0x55c410b946b0 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410c2b520 con 0x55c410d414a0 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410b0d8c0 con 0x55c410d41910 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.153+0000 7f52c0bb1b80 10 client.0 _ll_register_callbacks cb 0x55c410c49720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:39672/0 (socket says 172.21.15.6:39672) 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 -- 172.21.15.6:0/1850711722 learned_addr learned my addr 172.21.15.6:0/1850711722 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a659c700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 -- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 msgr2=0x55c410d4ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 -- 172.21.15.6:0/1850711722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 msgr2=0x55c410d4a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a759e700 1 -- 172.21.15.6:0/1850711722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a810e130 con 0x55c410d41910 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.154+0000 7f52a659c700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.155+0000 7f52a759e700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52a000a660 tx=0x7f52a000aa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a05c484a13ad7512 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.155+0000 7f528aff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.155+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0024040 con 0x55c410d41910 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.156+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c410d2b710 con 0x55c410d41910 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.156+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f52a0003e10 con 0x55c410d41910 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.156+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a00054c0 con 0x55c410d41910 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.156+0000 7f52c0bb1b80 10 client.26107 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.156+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c410c50130 con 0x55c410d41910 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.157+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c410d2b710 con 0x55c410d41910 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.157+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f52a001da60 con 0x55c410d41910 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.157+0000 7f52c0bb1b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.157+0000 7f52c0bb1b80 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52c0bb1b80 20 client.26107 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410d5a9c0 0x55c410d5cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52c0bb1b80 4 client.26107 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c410c4c040 con 0x55c410d5a9c0 2023-11-17T00:11:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410d5a9c0 0x55c410d5cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410d5a9c0 0x55c410d5cdd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55c4aec27083a08 server_cookie=9058f2ab3de60609 in_seq=0 out_seq=0 2023-11-17T00:11:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.158+0000 7f528aff5700 10 client.26107 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f528aff5700 1 -- 172.21.15.6:0/1850711722 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c410c4c040 con 0x55c410d5a9c0 2023-11-17T00:11:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f528aff5700 10 client.26107 handle_command_reply: tid=0 2023-11-17T00:11:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 client.26107 shutdown 2023-11-17T00:11:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 msgr2=0x55c410d4b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52a000a660 tx=0x7f52a000aa40 comp rx=0 tx=0).stop 2023-11-17T00:11:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 shutdown_connections 2023-11-17T00:11:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410d5a9c0 0x55c410d5cdd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d4b180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/1850711722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f528aff5700 0 client.26107 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 >> 172.21.15.6:0/1850711722 conn(0x55c410d2df50 msgr2=0x55c410d46fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.161+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 shutdown_connections 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.162+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/1850711722 wait complete. 2023-11-17T00:11:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.162+0000 7f52c0bb1b80 20 client.26107 trim_cache size 0 max 16384 2023-11-17T00:11:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 Processor -- start 2023-11-17T00:11:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 -- start start 2023-11-17T00:11:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d414a0 0x55c410d443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d41910 0x55c410d44910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410d41ce0 con 0x55c410b946b0 2023-11-17T00:11:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410d34c50 con 0x55c410d41910 2023-11-17T00:11:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410d48c60 con 0x55c410d414a0 2023-11-17T00:11:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.183+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45490/0 (socket says 172.21.15.6:45490) 2023-11-17T00:11:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 -- 172.21.15.6:0/3560216359 learned_addr learned my addr 172.21.15.6:0/3560216359 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 -- 172.21.15.6:0/3560216359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d414a0 msgr2=0x55c410d443d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d414a0 0x55c410d443d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 -- 172.21.15.6:0/3560216359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d41910 msgr2=0x55c410d44910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d41910 0x55c410d44910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 -- 172.21.15.6:0/3560216359 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c410d2f6f0 con 0x55c410b946b0 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.184+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f529800c5b0 tx=0x7f5298003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694e87c2202d3857 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.185+0000 7f528dffb700 1 -- 172.21.15.6:0/3560216359 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298099a80 con 0x55c410b946b0 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.185+0000 7f528dffb700 1 -- 172.21.15.6:0/3560216359 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5298096460 con 0x55c410b946b0 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.185+0000 7f528dffb700 1 -- 172.21.15.6:0/3560216359 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529809a1b0 con 0x55c410b946b0 2023-11-17T00:11:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/3560216359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 msgr2=0x55c410d51760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f529800c5b0 tx=0x7f5298003680 comp rx=0 tx=0).stop 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/3560216359 shutdown_connections 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d51760 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d41910 0x55c410d44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/3560216359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d414a0 0x55c410d443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/3560216359 >> 172.21.15.6:0/3560216359 conn(0x55c410d2df50 msgr2=0x55c410d55f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/3560216359 shutdown_connections 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/3560216359 wait complete. 2023-11-17T00:11:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.186+0000 7f52c0bb1b80 1 -- start start 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.187+0000 7f52c0bb1b80 1 Processor -- start 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410d51d50 con 0x55c410b946b0 2023-11-17T00:11:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410dad840 con 0x55c410d414a0 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c410dad980 con 0x55c410d41910 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52c0bb1b80 10 client.0 _ll_register_callbacks cb 0x7f52a000a240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52a6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45498/0 (socket says 172.21.15.6:45498) 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52a6d9d700 1 -- 172.21.15.6:0/4225007574 learned_addr learned my addr 172.21.15.6:0/4225007574 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.188+0000 7f52a659c700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a759e700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 -- 172.21.15.6:0/4225007574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 msgr2=0x55c410d501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 -- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 msgr2=0x55c410d4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 -- 172.21.15.6:0/4225007574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c410b0b5d0 con 0x55c410b946b0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a659c700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a759e700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f52a6d9d700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f529809a060 tx=0x7f5298099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a137e2be33bbf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.189+0000 7f528aff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529801e2e0 con 0x55c410b946b0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5298099ef0 con 0x55c410b946b0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f529809e150 con 0x55c410b946b0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52980a2570 con 0x55c410b946b0 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f52c0bb1b80 10 client.16071 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.190+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c410ab7d80 con 0x55c410b946b0 2023-11-17T00:11:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.191+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5298099ef0 con 0x55c410b946b0 2023-11-17T00:11:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.191+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f52980caa20 con 0x55c410b946b0 2023-11-17T00:11:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.191+0000 7f52c0bb1b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52c0bb1b80 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52c0bb1b80 20 client.16071 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410daebb0 0x55c410db0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52c0bb1b80 4 client.16071 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c410c58600 con 0x55c410daebb0 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.192+0000 7f52a659c700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410daebb0 0x55c410db0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.193+0000 7f52a659c700 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410daebb0 0x55c410db0f90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8afa7ab2d9336ad8 server_cookie=b6424c001c10c024 in_seq=0 out_seq=0 2023-11-17T00:11:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:21.193+0000 7f528aff5700 10 client.16071 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:22 smithi087 ceph-mon[131182]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:22.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:22 smithi186 ceph-mon[131487]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:22 smithi006 ceph-mon[97840]: pgmap v77: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.429+0000 7f528aff5700 1 -- 172.21.15.6:0/4225007574 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c410c58600 con 0x55c410daebb0 2023-11-17T00:11:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.429+0000 7f528aff5700 10 client.16071 handle_command_reply: tid=0 2023-11-17T00:11:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.429+0000 7f52c0bb1b80 1 client.16071 shutdown 2023-11-17T00:11:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.429+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 msgr2=0x55c410d4f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.429+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f529809a060 tx=0x7f5298099bb0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 shutdown_connections 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c410daebb0 0x55c410db0f90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c410b946b0 0x55c410d4f730 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c410d41910 0x55c410d501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f528aff5700 0 client.16071 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 --2- 172.21.15.6:0/4225007574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c410d414a0 0x55c410d4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 >> 172.21.15.6:0/4225007574 conn(0x55c410d2df50 msgr2=0x55c410d443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 shutdown_connections 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.431+0000 7f52c0bb1b80 1 -- 172.21.15.6:0/4225007574 wait complete. 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.432+0000 7f52c0bb1b80 20 client.16071 trim_cache size 0 max 16384 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:11:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.432+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 msgr2=0x7f52900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.432+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 0x7f52900a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52a000b780 tx=0x7f52a0005040 comp rx=0 tx=0).stop 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.432+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 msgr2=0x7f52a8198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 0x7f52a8198500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f529800e9c0 tx=0x7f529800ecd0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 shutdown_connections 2023-11-17T00:11:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f52900a12c0 0x7f52900a3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52a81048e0 0x7f52a819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a80fe920 0x7f52a8198500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 --2- 172.21.15.6:0/233325804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52a80ff2d0 0x7f52a8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 >> 172.21.15.6:0/233325804 conn(0x7f52a8074600 msgr2=0x7f52a80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 shutdown_connections 2023-11-17T00:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:23.433+0000 7f52ad1ff700 1 -- 172.21.15.6:0/233325804 wait complete. 2023-11-17T00:11:24.850 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:24 smithi087 ceph-mon[131182]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:24.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:24 smithi186 ceph-mon[131487]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:24.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:24 smithi006 ceph-mon[97840]: pgmap v78: 129 pgs: 129 active+clean; 12 MiB data, 97 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 805 KiB/s wr, 18 op/s 2023-11-17T00:11:25.854 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[156128]: *** Caught signal (Segmentation fault) ** 2023-11-17T00:11:25.855 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[156128]: in thread 7f45c104e700 thread_name:MR_Finisher 2023-11-17T00:11:26.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:26 smithi087 ceph-mon[131182]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:26.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:26 smithi186 ceph-mon[131487]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:26.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:26 smithi006 ceph-mon[97840]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:28.389 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:28 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service: Main process exited, code=exited, status=139/n/a 2023-11-17T00:11:28.536 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:28 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service: Failed with result 'exit-code'. 2023-11-17T00:11:28.536 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:28 smithi087 ceph-mon[131182]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:28.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:28 smithi186 ceph-mon[131487]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:28.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:28 smithi006 ceph-mon[97840]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 212 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 1.43k op/s 2023-11-17T00:11:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:30 smithi087 ceph-mon[131182]: pgmap v81: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.44k op/s 2023-11-17T00:11:30.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:30 smithi186 ceph-mon[131487]: pgmap v81: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.44k op/s 2023-11-17T00:11:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:30 smithi006 ceph-mon[97840]: pgmap v81: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.44k op/s 2023-11-17T00:11:32.854 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:32 smithi087 ceph-mon[131182]: pgmap v82: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:32.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:32 smithi186 ceph-mon[131487]: pgmap v82: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:32.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:32 smithi006 ceph-mon[97840]: pgmap v82: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:34 smithi087 ceph-mon[131182]: pgmap v83: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:34.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:34 smithi186 ceph-mon[131487]: pgmap v83: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:34.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:34 smithi006 ceph-mon[97840]: pgmap v83: 129 pgs: 129 active+clean; 16 MiB data, 223 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.9 MiB/s wr, 1.43k op/s 2023-11-17T00:11:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 -- 172.21.15.6:0/2490098834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 msgr2=0x7fd6580ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 --2- 172.21.15.6:0/2490098834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6580ff790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd64800dd20 tx=0x7fd64800f330 comp rx=0 tx=0).stop 2023-11-17T00:11:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 -- 172.21.15.6:0/2490098834 shutdown_connections 2023-11-17T00:11:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 --2- 172.21.15.6:0/2490098834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd658104400 0x7fd65810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 --2- 172.21.15.6:0/2490098834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6580ff790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 --2- 172.21.15.6:0/2490098834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6580ffd60 0x7fd658103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 -- 172.21.15.6:0/2490098834 >> 172.21.15.6:0/2490098834 conn(0x7fd6580763a0 msgr2=0x7fd6580767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 -- 172.21.15.6:0/2490098834 shutdown_connections 2023-11-17T00:11:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.614+0000 7fd65cfff700 1 -- 172.21.15.6:0/2490098834 wait complete. 2023-11-17T00:11:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.616+0000 7fd65cfff700 1 Processor -- start 2023-11-17T00:11:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.616+0000 7fd65cfff700 1 -- start start 2023-11-17T00:11:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6580ffd60 0x7fd658194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd658104400 0x7fd658198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581994b0 con 0x7fd658104400 2023-11-17T00:11:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd658199620 con 0x7fd6580ff3c0 2023-11-17T00:11:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65cfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581997e0 con 0x7fd6580ffd60 2023-11-17T00:11:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:53522/0 (socket says 172.21.15.6:53522) 2023-11-17T00:11:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd65659c700 1 -- 172.21.15.6:0/1015350260 learned_addr learned my addr 172.21.15.6:0/1015350260 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.617+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6580ffd60 0x7fd658194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 -- 172.21.15.6:0/1015350260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6580ffd60 msgr2=0x7fd658194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6580ffd60 0x7fd658194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 -- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd658104400 msgr2=0x7fd658198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd658104400 0x7fd658198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 -- 172.21.15.6:0/1015350260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64800da00 con 0x7fd6580ff3c0 2023-11-17T00:11:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65659c700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd64800f7b0 tx=0x7fd64800fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd6477fe700 1 -- 172.21.15.6:0/1015350260 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648012040 con 0x7fd6580ff3c0 2023-11-17T00:11:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd6477fe700 1 -- 172.21.15.6:0/1015350260 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd648005600 con 0x7fd6580ff3c0 2023-11-17T00:11:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65810e1e0 con 0x7fd6580ff3c0 2023-11-17T00:11:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.618+0000 7fd6477fe700 1 -- 172.21.15.6:0/1015350260 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800bc10 con 0x7fd6580ff3c0 2023-11-17T00:11:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.619+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65810e050 con 0x7fd6580ff3c0 2023-11-17T00:11:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.624+0000 7fd6477fe700 1 -- 172.21.15.6:0/1015350260 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd648003b60 con 0x7fd6580ff3c0 2023-11-17T00:11:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.624+0000 7fd6709abb80 1 Processor -- start 2023-11-17T00:11:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.624+0000 7fd6709abb80 1 -- start start 2023-11-17T00:11:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e3d2000 con 0x55ea9e4d2b90 2023-11-17T00:11:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e424840 con 0x55ea9e4d2720 2023-11-17T00:11:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd6709abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e41dcd0 con 0x55ea9e3286b0 2023-11-17T00:11:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:53526/0 (socket says 172.21.15.6:53526) 2023-11-17T00:11:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd65659c700 1 -- 172.21.15.6:0/3357080001 learned_addr learned my addr 172.21.15.6:0/3357080001 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd655d9b700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.625+0000 7fd656d9d700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd6477fe700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 0x7fd6400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 -- 172.21.15.6:0/3357080001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 msgr2=0x55ea9e4d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 -- 172.21.15.6:0/3357080001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 msgr2=0x55ea9e4cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 -- 172.21.15.6:0/3357080001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea9e29f5d0 con 0x55ea9e4d2b90 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd65659c700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd6477fe700 1 -- 172.21.15.6:0/1015350260 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd6480d1780 con 0x7fd6580ff3c0 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.626+0000 7fd655d9b700 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ccb00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd65806a3c0 tx=0x7fd64c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e549456c45b8128b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.627+0000 7fd644ff9700 1 -- 172.21.15.6:0/3357080001 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c011590 con 0x55ea9e4d2b90 2023-11-17T00:11:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.627+0000 7fd644ff9700 1 -- 172.21.15.6:0/3357080001 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64c011bb0 con 0x55ea9e4d2b90 2023-11-17T00:11:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.627+0000 7fd644ff9700 1 -- 172.21.15.6:0/3357080001 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c012ca0 con 0x55ea9e4d2b90 2023-11-17T00:11:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.627+0000 7fd656d9d700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 0x7fd6400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.628+0000 7fd6709abb80 1 -- 172.21.15.6:0/3357080001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 msgr2=0x55ea9e4ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.628+0000 7fd6709abb80 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ccb00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd65806a3c0 tx=0x7fd64c007a00 comp rx=0 tx=0).stop 2023-11-17T00:11:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd656d9d700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 0x7fd6400a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e80 tx=0x7fd650004be0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 -- 172.21.15.6:0/3357080001 shutdown_connections 2023-11-17T00:11:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ccb00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 --2- 172.21.15.6:0/3357080001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 -- 172.21.15.6:0/3357080001 >> 172.21.15.6:0/3357080001 conn(0x55ea9e4c1f50 msgr2=0x55ea9e4c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 -- 172.21.15.6:0/3357080001 shutdown_connections 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 -- 172.21.15.6:0/3357080001 wait complete. 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.629+0000 7fd6709abb80 1 -- start start 2023-11-17T00:11:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 Processor -- start 2023-11-17T00:11:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e3bf880 con 0x55ea9e4d2b90 2023-11-17T00:11:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e3bf520 con 0x55ea9e4d2720 2023-11-17T00:11:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e2a18c0 con 0x55ea9e3286b0 2023-11-17T00:11:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.630+0000 7fd6709abb80 10 client.0 _ll_register_callbacks cb 0x55ea9e3dd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51876/0 (socket says 172.21.15.6:51876) 2023-11-17T00:11:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 -- 172.21.15.6:0/94430602 learned_addr learned my addr 172.21.15.6:0/94430602 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd656d9d700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 -- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 msgr2=0x55ea9e4d5c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:11:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d5c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 -- 172.21.15.6:0/94430602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 msgr2=0x55ea9e4d62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4d62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 -- 172.21.15.6:0/94430602 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c00e040 con 0x55ea9e3286b0 2023-11-17T00:11:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd65659c700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6480a63c0 tx=0x7fd64809a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb608d6a7260b4f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.631+0000 7fd63dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:11:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.632+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64809de90 con 0x55ea9e3286b0 2023-11-17T00:11:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.632+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd64809e530 con 0x55ea9e3286b0 2023-11-17T00:11:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.632+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea9e4bf710 con 0x55ea9e3286b0 2023-11-17T00:11:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.633+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6480a2560 con 0x55ea9e3286b0 2023-11-17T00:11:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.633+0000 7fd6709abb80 10 client.26116 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.633+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ea9e3e4130 con 0x55ea9e3286b0 2023-11-17T00:11:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.633+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea9e4bf710 con 0x55ea9e3286b0 2023-11-17T00:11:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6480d1340 con 0x55ea9e3286b0 2023-11-17T00:11:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 10 client.26116 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 20 client.26116 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e4e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 4 client.26116 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.634+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea9e3e0040 con 0x55ea9e4e5cc0 2023-11-17T00:11:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.635+0000 7fd655d9b700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e4e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.635+0000 7fd655d9b700 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e4e80d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abae1ca9b90e539 server_cookie=66bd1022f1ea44bf in_seq=0 out_seq=0 2023-11-17T00:11:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.635+0000 7fd63dffb700 10 client.26116 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd63dffb700 1 -- 172.21.15.6:0/94430602 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea9e3e0040 con 0x55ea9e4e5cc0 2023-11-17T00:11:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd63dffb700 10 client.26116 handle_command_reply: tid=0 2023-11-17T00:11:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 client.26116 shutdown 2023-11-17T00:11:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 msgr2=0x55ea9e4d8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6480a63c0 tx=0x7fd64809a920 comp rx=0 tx=0).stop 2023-11-17T00:11:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 shutdown_connections 2023-11-17T00:11:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e4e80d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd63dffb700 0 client.26116 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e3286b0 0x55ea9e4d8a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 --2- 172.21.15.6:0/94430602 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.637+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 >> 172.21.15.6:0/94430602 conn(0x55ea9e4c1f50 msgr2=0x55ea9e4c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.638+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 shutdown_connections 2023-11-17T00:11:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.638+0000 7fd6709abb80 1 -- 172.21.15.6:0/94430602 wait complete. 2023-11-17T00:11:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.638+0000 7fd6709abb80 20 client.26116 trim_cache size 0 max 16384 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.660+0000 7fd6709abb80 1 Processor -- start 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.660+0000 7fd6709abb80 1 -- start start 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4c8a00 con 0x55ea9e4d2b90 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4d4210 con 0x55ea9e3286b0 2023-11-17T00:11:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd6709abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4e1f50 con 0x55ea9e4d2720 2023-11-17T00:11:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.661+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51878/0 (socket says 172.21.15.6:51878) 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 -- 172.21.15.6:0/1966341291 learned_addr learned my addr 172.21.15.6:0/1966341291 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd656d9d700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd65659c700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 -- 172.21.15.6:0/1966341291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 msgr2=0x55ea9e4dd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4dd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 -- 172.21.15.6:0/1966341291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 msgr2=0x55ea9e4c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 -- 172.21.15.6:0/1966341291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea9e4c36f0 con 0x55ea9e4d2720 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd65659c700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.662+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0048a0 tx=0x7fd64c012880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edec25e21da95f83 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.663+0000 7fd6457fa700 1 -- 172.21.15.6:0/1966341291 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c012420 con 0x55ea9e4d2720 2023-11-17T00:11:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.663+0000 7fd6457fa700 1 -- 172.21.15.6:0/1966341291 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64c029070 con 0x55ea9e4d2720 2023-11-17T00:11:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.663+0000 7fd6457fa700 1 -- 172.21.15.6:0/1966341291 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c012be0 con 0x55ea9e4d2720 2023-11-17T00:11:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 -- 172.21.15.6:0/1966341291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 msgr2=0x55ea9e4d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0048a0 tx=0x7fd64c012880 comp rx=0 tx=0).stop 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 -- 172.21.15.6:0/1966341291 shutdown_connections 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1966341291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4d36a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 -- 172.21.15.6:0/1966341291 >> 172.21.15.6:0/1966341291 conn(0x55ea9e4c1f50 msgr2=0x55ea9e4d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 -- 172.21.15.6:0/1966341291 shutdown_connections 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.664+0000 7fd6709abb80 1 -- 172.21.15.6:0/1966341291 wait complete. 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.665+0000 7fd6709abb80 1 -- start start 2023-11-17T00:11:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.665+0000 7fd6709abb80 1 Processor -- start 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4cec20 con 0x55ea9e4d2b90 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4ced60 con 0x55ea9e3286b0 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9e4cef20 con 0x55ea9e4d2720 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd6709abb80 10 client.0 _ll_register_callbacks cb 0x7fd64800d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:53558/0 (socket says 172.21.15.6:53558) 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 -- 172.21.15.6:0/1755023749 learned_addr learned my addr 172.21.15.6:0/1755023749 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd656d9d700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 -- 172.21.15.6:0/1755023749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 msgr2=0x55ea9e4ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.666+0000 7fd65659c700 1 -- 172.21.15.6:0/1755023749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 msgr2=0x55ea9e4ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd65659c700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd65659c700 1 -- 172.21.15.6:0/1755023749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea9e29f5d0 con 0x55ea9e3286b0 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd656d9d700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd65659c700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd64809c880 tx=0x7fd6480cee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da31d08a10205631 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd63dffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64809e790 con 0x55ea9e3286b0 2023-11-17T00:11:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.667+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd6480d7220 con 0x55ea9e3286b0 2023-11-17T00:11:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.668+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd64c011bb0 con 0x55ea9e3286b0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.668+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64809e530 con 0x55ea9e3286b0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.668+0000 7fd6709abb80 10 client.26141 fetch_fsmap learned FSMap version 20 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.668+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ea9e24bd80 con 0x55ea9e3286b0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.668+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64c011bb0 con 0x55ea9e3286b0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd64809da20 con 0x55ea9e3286b0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 10 client.26141 fetch_fsmap finished waiting for FSMap version 20 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 20 client.26141 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e544aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 4 client.26141 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea9e3ec600 con 0x55ea9e4e5cc0 2023-11-17T00:11:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.669+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e544aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.670+0000 7fd655d9b700 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e544aa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e6b08fc3097123 server_cookie=c58b9ae90aa35d79 in_seq=0 out_seq=0 2023-11-17T00:11:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:36.670+0000 7fd63dffb700 10 client.26141 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:36 smithi087 ceph-mon[131182]: pgmap v84: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.7 MiB/s wr, 1.43k op/s 2023-11-17T00:11:36.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:36 smithi186 ceph-mon[131487]: pgmap v84: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.7 MiB/s wr, 1.43k op/s 2023-11-17T00:11:36.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:36 smithi006 ceph-mon[97840]: pgmap v84: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.7 MiB/s wr, 1.43k op/s 2023-11-17T00:11:38.851 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:38 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service: Service RestartSec=10s expired, scheduling restart. 2023-11-17T00:11:38.851 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:38 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service: Scheduled restart job, restart counter is at 1. 2023-11-17T00:11:38.851 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:38 smithi087 systemd[1]: Stopped Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:11:38.851 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:38 smithi087 systemd[1]: Starting Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:11:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:38 smithi087 ceph-mon[131182]: pgmap v85: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:38 smithi087 ceph-mon[131182]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2023-11-17T00:11:38.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:38 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 9 up:standby 2023-11-17T00:11:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:38 smithi186 ceph-mon[131487]: pgmap v85: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:38.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:38 smithi186 ceph-mon[131487]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2023-11-17T00:11:38.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:38 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 9 up:standby 2023-11-17T00:11:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:38 smithi006 ceph-mon[97840]: pgmap v85: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:38 smithi006 ceph-mon[97840]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2023-11-17T00:11:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:38 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 9 up:standby 2023-11-17T00:11:39.602 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:39 smithi087 podman[160421]: 2023-11-17T00:11:40.102 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:39 smithi087 bash[160421]: a34d274c673df44b38cde80ca9f645f4bb5342c7b7a33500b88faacdf79e5d62 2023-11-17T00:11:40.102 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[160432]: starting mds.h at 2023-11-17T00:11:40.102 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:11:39 smithi087 systemd[1]: Started Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:11:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:40 smithi006 ceph-mon[97840]: pgmap v86: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:40 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6836/1614644612,v1:172.21.15.87:6837/1614644612] up:boot 2023-11-17T00:11:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:40 smithi006 ceph-mon[97840]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:11:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:40 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:11:41.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:40 smithi087 ceph-mon[131182]: pgmap v86: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:41.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:40 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6836/1614644612,v1:172.21.15.87:6837/1614644612] up:boot 2023-11-17T00:11:41.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:40 smithi087 ceph-mon[131182]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:11:41.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:40 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:11:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:40 smithi186 ceph-mon[131487]: pgmap v86: 129 pgs: 129 active+clean; 25 MiB data, 230 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.7 MiB/s wr, 16 op/s 2023-11-17T00:11:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:40 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6836/1614644612,v1:172.21.15.87:6837/1614644612] up:boot 2023-11-17T00:11:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:40 smithi186 ceph-mon[131487]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-11-17T00:11:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:40 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-17T00:11:41.966 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:11:42.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.574+0000 7f4a049ff700 1 -- 172.21.15.6:0/3800757054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a001015f0 msgr2=0x7f4a00104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:42.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.574+0000 7f4a049ff700 1 --2- 172.21.15.6:0/3800757054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a001015f0 0x7f4a00104e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49f00099c0 tx=0x7f49f0009cd0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/3800757054 shutdown_connections 2023-11-17T00:11:42.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/3800757054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a00100c50 0x7f4a00101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/3800757054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00105560 0x7f4a0010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/3800757054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a001015f0 0x7f4a00104e80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/3800757054 >> 172.21.15.6:0/3800757054 conn(0x7f4a00077500 msgr2=0x7f4a00077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:42.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.576+0000 7f4a049ff700 1 -- 172.21.15.6:0/3800757054 shutdown_connections 2023-11-17T00:11:42.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.577+0000 7f4a049ff700 1 -- 172.21.15.6:0/3800757054 wait complete. 2023-11-17T00:11:42.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.579+0000 7f4a049ff700 1 Processor -- start 2023-11-17T00:11:42.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 -- start start 2023-11-17T00:11:42.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 0x7f4a0010c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 0x7f4a0010c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 0x7f4a001b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001b1670 con 0x7f4a001015f0 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001b17b0 con 0x7f4a00105560 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.580+0000 7f4a049ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001b1920 con 0x7f4a00100c50 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.581+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 0x7f4a0010c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.581+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 0x7f4a0010c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51218/0 (socket says 172.21.15.6:51218) 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.581+0000 7f49fe59c700 1 -- 172.21.15.6:0/2326751389 learned_addr learned my addr 172.21.15.6:0/2326751389 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:42.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.581+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 0x7f4a0010c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.581+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 0x7f4a001b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fed9d700 1 -- 172.21.15.6:0/2326751389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 msgr2=0x7f4a0010c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 0x7f4a0010c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fed9d700 1 -- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 msgr2=0x7f4a0010c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 0x7f4a0010c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fed9d700 1 -- 172.21.15.6:0/2326751389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00095f0 con 0x7f4a00105560 2023-11-17T00:11:42.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.582+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 0x7f4a0010c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:42.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.583+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 0x7f4a001b1130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007f10 tx=0x7f49f800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:42.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.583+0000 7f49ef7fe700 1 -- 172.21.15.6:0/2326751389 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800ddd0 con 0x7f4a00105560 2023-11-17T00:11:42.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.584+0000 7f49ef7fe700 1 -- 172.21.15.6:0/2326751389 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f800f040 con 0x7f4a00105560 2023-11-17T00:11:42.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.584+0000 7f49ef7fe700 1 -- 172.21.15.6:0/2326751389 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8005a90 con 0x7f4a00105560 2023-11-17T00:11:42.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.584+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a00110410 con 0x7f4a00105560 2023-11-17T00:11:42.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.584+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a00110280 con 0x7f4a00105560 2023-11-17T00:11:42.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.596+0000 7f49ef7fe700 1 -- 172.21.15.6:0/2326751389 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f49f8004ad0 con 0x7f4a00105560 2023-11-17T00:11:42.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.597+0000 7f4a182c3b80 1 Processor -- start 2023-11-17T00:11:42.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.600+0000 7f49ef7fe700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 0x7f49e80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.600+0000 7f49ef7fe700 1 -- 172.21.15.6:0/2326751389 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49f80cc860 con 0x7f4a00105560 2023-11-17T00:11:42.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.602+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 0x7f49e80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 -- start start 2023-11-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 0x5580e3fa2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 0x5580e3fa7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3fa7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3ea8060 con 0x5580e3fa2b60 2023-11-17T00:11:42.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.603+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3efa8a0 con 0x5580e3fa78b0 2023-11-17T00:11:42.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.604+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3ef3d30 con 0x5580e3fa3470 2023-11-17T00:11:42.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.604+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 0x5580e3fa7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.604+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 0x5580e3fa7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51224/0 (socket says 172.21.15.6:51224) 2023-11-17T00:11:42.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.604+0000 7f49fdd9b700 1 -- 172.21.15.6:0/4153065135 learned_addr learned my addr 172.21.15.6:0/4153065135 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.605+0000 7f49fed9d700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3fa7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.605+0000 7f49fe59c700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 0x5580e3fa2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.605+0000 7f49fed9d700 1 -- 172.21.15.6:0/4153065135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 msgr2=0x5580e3fa7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.605+0000 7f49fed9d700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 0x5580e3fa7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.605+0000 7f49fed9d700 1 -- 172.21.15.6:0/4153065135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 msgr2=0x5580e3fa2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.606+0000 7f49fed9d700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 0x5580e3fa2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.606+0000 7f49fed9d700 1 -- 172.21.15.6:0/4153065135 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e3d75630 con 0x5580e3fa78b0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.606+0000 7f49fe59c700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 0x5580e3fa2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.606+0000 7f49fed9d700 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3fa7d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49f80c9c00 tx=0x7f49f8099140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfdbaeae201acb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.606+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 0x7f49e80a38b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a001019c0 tx=0x7f49f0000bc0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.607+0000 7f49ecff9700 1 -- 172.21.15.6:0/4153065135 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8099580 con 0x5580e3fa78b0 2023-11-17T00:11:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.607+0000 7f49ecff9700 1 -- 172.21.15.6:0/4153065135 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f809d5b0 con 0x5580e3fa78b0 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.607+0000 7f49ecff9700 1 -- 172.21.15.6:0/4153065135 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80cc450 con 0x5580e3fa78b0 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.608+0000 7f4a182c3b80 1 -- 172.21.15.6:0/4153065135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 msgr2=0x5580e3fa7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.608+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3fa7d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49f80c9c00 tx=0x7f49f8099140 comp rx=0 tx=0).stop 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 -- 172.21.15.6:0/4153065135 shutdown_connections 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa2b60 0x5580e3fa2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3fa7d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/4153065135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa3470 0x5580e3fa7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 -- 172.21.15.6:0/4153065135 >> 172.21.15.6:0/4153065135 conn(0x5580e3f98360 msgr2=0x5580e3f9db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 -- 172.21.15.6:0/4153065135 shutdown_connections 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.609+0000 7f4a182c3b80 1 -- 172.21.15.6:0/4153065135 wait complete. 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.610+0000 7f4a182c3b80 1 -- start start 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.611+0000 7f4a182c3b80 1 Processor -- start 2023-11-17T00:11:42.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.611+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3faf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.612+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.612+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.612+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39074/0 (socket says 172.21.15.6:39074) 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.612+0000 7f49fdd9b700 1 -- 172.21.15.6:0/2923520459 learned_addr learned my addr 172.21.15.6:0/2923520459 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.612+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3fac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.613+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3fac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.613+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3e958e0 con 0x5580e3fa3470 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.613+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3faf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.613+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3e95580 con 0x5580e3fa2b60 2023-11-17T00:11:42.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.613+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3d77920 con 0x5580e3fa78b0 2023-11-17T00:11:42.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.614+0000 7f4a182c3b80 10 client.0 _ll_register_callbacks cb 0x5580e3eb3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:42.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.615+0000 7f49fe59c700 1 -- 172.21.15.6:0/2923520459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 msgr2=0x5580e3fac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:42.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.615+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3fac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.615+0000 7f49fe59c700 1 -- 172.21.15.6:0/2923520459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 msgr2=0x5580e3fac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.615+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.615+0000 7f49fe59c700 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a00110280 con 0x5580e3fa2b60 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.616+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3faf010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f49f4010470 tx=0x7f49f4010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39f1845756bedbd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.617+0000 7f49e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.617+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4014800 con 0x5580e3fa2b60 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.618+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49f4014960 con 0x5580e3fa2b60 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.618+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580e3f95b20 con 0x5580e3fa2b60 2023-11-17T00:11:42.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.620+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f4014b20 con 0x5580e3fa2b60 2023-11-17T00:11:42.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.620+0000 7f4a182c3b80 10 client.26159 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:42.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.620+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5580e3eba190 con 0x5580e3fa2b60 2023-11-17T00:11:42.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.621+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580e3f95b20 con 0x5580e3fa2b60 2023-11-17T00:11:42.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.622+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49f401c3b0 con 0x5580e3fa2b60 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 10 client.26159 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 20 client.26159 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e3fbebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 4 client.26159 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580e3eb60a0 con 0x5580e3fbc7a0 2023-11-17T00:11:42.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.623+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e3fbebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:42.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.624+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e3fbebb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5570151f0fd198 server_cookie=54d46848b6c4ddb7 in_seq=0 out_seq=0 2023-11-17T00:11:42.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:42.626+0000 7f49e5ffb700 10 client.26159 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:42.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:42 smithi006 ceph-mon[97840]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:42 smithi087 ceph-mon[131182]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:42 smithi186 ceph-mon[131487]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.531+0000 7fd63dffb700 1 -- 172.21.15.6:0/1755023749 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea9e3ec600 con 0x55ea9e4e5cc0 2023-11-17T00:11:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.531+0000 7fd63dffb700 10 client.26141 handle_command_reply: tid=0 2023-11-17T00:11:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.531+0000 7fd6709abb80 1 client.26141 shutdown 2023-11-17T00:11:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.531+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 msgr2=0x55ea9e4cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.531+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd64809c880 tx=0x7fd6480cee50 comp rx=0 tx=0).stop 2023-11-17T00:11:43.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.531+0000 7f49e5ffb700 1 -- 172.21.15.6:0/2923520459 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580e3eb60a0 con 0x5580e3fbc7a0 2023-11-17T00:11:43.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.531+0000 7f49e5ffb700 10 client.26159 handle_command_reply: tid=0 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 shutdown_connections 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ea9e4e5cc0 0x55ea9e544aa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ea9e4d2b90 0x55ea9e4ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ea9e3286b0 0x55ea9e4cdc60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 --2- 172.21.15.6:0/1755023749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ea9e4d2720 0x55ea9e4ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 >> 172.21.15.6:0/1755023749 conn(0x55ea9e4c1f50 msgr2=0x55ea9e4d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.532+0000 7f4a182c3b80 1 client.26159 shutdown 2023-11-17T00:11:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd63dffb700 0 client.26141 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 shutdown_connections 2023-11-17T00:11:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 1 -- 172.21.15.6:0/1755023749 wait complete. 2023-11-17T00:11:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.532+0000 7fd6709abb80 20 client.26141 trim_cache size 0 max 16384 2023-11-17T00:11:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:11:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:11:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:11:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:11:43.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 msgr2=0x5580e3faf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3faf010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f49f4010470 tx=0x7f49f4010830 comp rx=0 tx=0).stop 2023-11-17T00:11:43.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 shutdown_connections 2023-11-17T00:11:43.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3fac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3faf010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e3fbebb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.533+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 msgr2=0x7fd6400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.533+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 0x7fd6400a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e80 tx=0x7fd650004be0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.533+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 msgr2=0x7fd6581944d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.533+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd64800f7b0 tx=0x7fd64800fd90 comp rx=0 tx=0).stop 2023-11-17T00:11:43.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2923520459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f49e5ffb700 0 client.26159 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:43.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.533+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 >> 172.21.15.6:0/2923520459 conn(0x5580e3f98360 msgr2=0x5580e3fa8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.534+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 shutdown_connections 2023-11-17T00:11:43.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.534+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2923520459 wait complete. 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 shutdown_connections 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd6400a1340 0x7fd6400a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd658104400 0x7fd658198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6580ff3c0 0x7fd6581944d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 --2- 172.21.15.6:0/1015350260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6580ffd60 0x7fd658194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 >> 172.21.15.6:0/1015350260 conn(0x7fd6580763a0 msgr2=0x7fd658102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.534+0000 7f4a182c3b80 20 client.26159 trim_cache size 0 max 16384 2023-11-17T00:11:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 shutdown_connections 2023-11-17T00:11:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:43.534+0000 7fd65cfff700 1 -- 172.21.15.6:0/1015350260 wait complete. 2023-11-17T00:11:43.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.561+0000 7f4a182c3b80 1 Processor -- start 2023-11-17T00:11:43.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.561+0000 7f4a182c3b80 1 -- start start 2023-11-17T00:11:43.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fa8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51244/0 (socket says 172.21.15.6:51244) 2023-11-17T00:11:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fa8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f49fdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fa8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39076/0 (socket says 172.21.15.6:39076) 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f49fe59c700 1 -- 172.21.15.6:0/2361481398 learned_addr learned my addr 172.21.15.6:0/2361481398 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3faebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3f9f430 con 0x5580e3fa3470 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3fa3950 con 0x5580e3fa78b0 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.562+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3fb88c0 con 0x5580e3fa2b60 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fed9d700 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3faebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 -- 172.21.15.6:0/2361481398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 msgr2=0x5580e3faebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3faebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 -- 172.21.15.6:0/2361481398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 msgr2=0x5580e3fa8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fa8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 -- 172.21.15.6:0/2361481398 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e3eb9ee0 con 0x5580e3fa2b60 2023-11-17T00:11:43.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49fe59c700 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009420 tx=0x7f49f401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18eb24ed881c1c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:43.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.563+0000 7f49ed7fa700 1 -- 172.21.15.6:0/2361481398 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4006e60 con 0x5580e3fa2b60 2023-11-17T00:11:43.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.564+0000 7f49ed7fa700 1 -- 172.21.15.6:0/2361481398 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f401ca30 con 0x5580e3fa2b60 2023-11-17T00:11:43.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.564+0000 7f49ed7fa700 1 -- 172.21.15.6:0/2361481398 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4014a20 con 0x5580e3fa2b60 2023-11-17T00:11:43.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.566+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 msgr2=0x5580e3fb3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.566+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009420 tx=0x7f49f401c8e0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 shutdown_connections 2023-11-17T00:11:43.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa78b0 0x5580e3faebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa2b60 0x5580e3fb3a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/2361481398 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fa8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 >> 172.21.15.6:0/2361481398 conn(0x5580e3f98360 msgr2=0x5580e3fb81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 shutdown_connections 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 -- 172.21.15.6:0/2361481398 wait complete. 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.567+0000 7f4a182c3b80 1 -- start start 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.568+0000 7f4a182c3b80 1 Processor -- start 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3fabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3faaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3fab040 con 0x5580e3fa3470 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3fab1b0 con 0x5580e3fa2b60 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3fab370 con 0x5580e3fa78b0 2023-11-17T00:11:43.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.569+0000 7f4a182c3b80 10 client.0 _ll_register_callbacks cb 0x7f49f4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3faaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3faaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51252/0 (socket says 172.21.15.6:51252) 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fed9d700 1 -- 172.21.15.6:0/1333971139 learned_addr learned my addr 172.21.15.6:0/1333971139 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3fabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 -- 172.21.15.6:0/1333971139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 msgr2=0x5580e3faaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3faaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 -- 172.21.15.6:0/1333971139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 msgr2=0x5580e3fac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fe59c700 1 -- 172.21.15.6:0/1333971139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580e3eba190 con 0x5580e3fa2b60 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.570+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.571+0000 7f49fe59c700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3fabe10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49f401c740 tx=0x7f49f401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d930c1aabb35ebc server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:43.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.571+0000 7f49e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.571+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4037070 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.571+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49f4005db0 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.571+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49f809d5b0 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f403a710 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 10 client.26165 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5580e3d0ac60 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f809d5b0 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49f403a030 con 0x5580e3fa2b60 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 20 client.26165 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e4013180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 4 client.26165 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.572+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5580e3ec2660 con 0x5580e3fbc7a0 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.573+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e4013180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.573+0000 7f49fdd9b700 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e4013180 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acddf8a18e8e7df server_cookie=8bd4d401540467e9 in_seq=0 out_seq=0 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.573+0000 7f49e5ffb700 10 client.26165 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f49e5ffb700 1 -- 172.21.15.6:0/1333971139 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5580e3ec2660 con 0x5580e3fbc7a0 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f49e5ffb700 10 client.26165 handle_command_reply: tid=0 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 client.26165 shutdown 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 msgr2=0x5580e3fabe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3fabe10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49f401c740 tx=0x7f49f401c950 comp rx=0 tx=0).stop 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 shutdown_connections 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5580e3fa78b0 0x5580e3faaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5580e3fa2b60 0x5580e3fabe10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5580e3fbc7a0 0x5580e4013180 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 --2- 172.21.15.6:0/1333971139 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5580e3fa3470 0x5580e3fac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 >> 172.21.15.6:0/1333971139 conn(0x5580e3f98360 msgr2=0x5580e3fa9570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f49e5ffb700 0 client.26165 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 shutdown_connections 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.574+0000 7f4a182c3b80 1 -- 172.21.15.6:0/1333971139 wait complete. 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a182c3b80 20 client.26165 trim_cache size 0 max 16384 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:11:43.584 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 msgr2=0x7f49e80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 0x7f49e80a38b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a001019c0 tx=0x7f49f0000bc0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 msgr2=0x7f4a001b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 0x7f4a001b1130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007f10 tx=0x7f49f800d3b0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 shutdown_connections 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f49e80a1400 0x7f49e80a38b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a00105560 0x7f4a001b1130 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a00100c50 0x7f4a0010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 --2- 172.21.15.6:0/2326751389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a001015f0 0x7f4a0010c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 >> 172.21.15.6:0/2326751389 conn(0x7f4a00077500 msgr2=0x7f4a000fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.575+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 shutdown_connections 2023-11-17T00:11:43.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.576+0000 7f4a049ff700 1 -- 172.21.15.6:0/2326751389 wait complete. 2023-11-17T00:11:43.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:11:43.602 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:11:43.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.851+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/2510583938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 msgr2=0x7fcb24101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.851+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/2510583938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb24101090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb200099c0 tx=0x7fcb20009cd0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.852+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/2510583938 shutdown_connections 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.852+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/2510583938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb24105620 0x7fcb2410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.852+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/2510583938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb24101090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.852+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/2510583938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24101660 0x7fcb24104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.852+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/2510583938 >> 172.21.15.6:0/2510583938 conn(0x7fcb240774d0 msgr2=0x7fcb240778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:43.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.853+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/2510583938 shutdown_connections 2023-11-17T00:11:43.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.853+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/2510583938 wait complete. 2023-11-17T00:11:43.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.854+0000 7fcb2b9ff700 1 Processor -- start 2023-11-17T00:11:43.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 -- start start 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb24101660 0x7fcb24198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 0x7fcb2419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c5a0 con 0x7fcb24105620 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c710 con 0x7fcb24100cc0 2023-11-17T00:11:43.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2b9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419c8d0 con 0x7fcb24101660 2023-11-17T00:11:43.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.855+0000 7fcb2979b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55752/0 (socket says 172.21.15.6:55752) 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 -- 172.21.15.6:0/3992074786 learned_addr learned my addr 172.21.15.6:0/3992074786 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 -- 172.21.15.6:0/3992074786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb24101660 msgr2=0x7fcb24198bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb24101660 0x7fcb24198bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 -- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 msgr2=0x7fcb2419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb29f9c700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 0x7fcb2419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 0x7fcb2419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 -- 172.21.15.6:0/3992074786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200095f0 con 0x7fcb24100cc0 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb29f9c700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 0x7fcb2419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.856+0000 7fcb2979b700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b580 tx=0x7fcb2000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.857+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2001d070 con 0x7fcb24100cc0 2023-11-17T00:11:43.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.857+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb20005510 con 0x7fcb24100cc0 2023-11-17T00:11:43.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.857+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20004bd0 con 0x7fcb24100cc0 2023-11-17T00:11:43.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.857+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb241103d0 con 0x7fcb24100cc0 2023-11-17T00:11:43.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.857+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb24110240 con 0x7fcb24100cc0 2023-11-17T00:11:43.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.859+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08005310 con 0x7fcb24100cc0 2023-11-17T00:11:43.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.862+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcb20004d30 con 0x7fcb24100cc0 2023-11-17T00:11:43.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.863+0000 7fcb1a7fc700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 0x7fcb100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:43.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.864+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcb200cdde0 con 0x7fcb24100cc0 2023-11-17T00:11:43.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.864+0000 7fcb28f9a700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 0x7fcb100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:43.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.864+0000 7fcb28f9a700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 0x7fcb100a37f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb24199cc0 tx=0x7fcb14009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:43.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:43.867+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb20096ed0 con 0x7fcb24100cc0 2023-11-17T00:11:44.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.230+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb080059e0 con 0x7fcb24100cc0 2023-11-17T00:11:44.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.231+0000 7fcb1a7fc700 1 -- 172.21.15.6:0/3992074786 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fcb2009a4f0 con 0x7fcb24100cc0 2023-11-17T00:11:44.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:11:44.240 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 msgr2=0x7fcb100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 0x7fcb100a37f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb24199cc0 tx=0x7fcb14009260 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 msgr2=0x7fcb241986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b580 tx=0x7fcb2000bfd0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 shutdown_connections 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb100a1340 0x7fcb100a37f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb24105620 0x7fcb2419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb24100cc0 0x7fcb241986b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 --2- 172.21.15.6:0/3992074786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb24101660 0x7fcb24198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.233+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 >> 172.21.15.6:0/3992074786 conn(0x7fcb240774d0 msgr2=0x7fcb240fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.234+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 shutdown_connections 2023-11-17T00:11:44.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.234+0000 7fcb2b9ff700 1 -- 172.21.15.6:0/3992074786 wait complete. 2023-11-17T00:11:44.242 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:11:44.258 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:11:44.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 -- 172.21.15.6:0/660331042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0105250 msgr2=0x7f28e0105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 --2- 172.21.15.6:0/660331042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0105250 0x7f28e0105620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009990 tx=0x7f28dc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 -- 172.21.15.6:0/660331042 shutdown_connections 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 --2- 172.21.15.6:0/660331042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0068770 0x7f28e010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 --2- 172.21.15.6:0/660331042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0105250 0x7f28e0105620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 --2- 172.21.15.6:0/660331042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 0x7f28e0068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.509+0000 7f28e87ff700 1 -- 172.21.15.6:0/660331042 >> 172.21.15.6:0/660331042 conn(0x7f28e00763a0 msgr2=0x7f28e00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.510+0000 7f28e87ff700 1 -- 172.21.15.6:0/660331042 shutdown_connections 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.510+0000 7f28e87ff700 1 -- 172.21.15.6:0/660331042 wait complete. 2023-11-17T00:11:44.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 Processor -- start 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 -- start start 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 0x7f28e0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0105250 0x7f28e019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019d630 con 0x7f28e0067dc0 2023-11-17T00:11:44.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019d7a0 con 0x7f28e0068770 2023-11-17T00:11:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.511+0000 7f28e87ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e019d960 con 0x7f28e0105250 2023-11-17T00:11:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55780/0 (socket says 172.21.15.6:55780) 2023-11-17T00:11:44.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 -- 172.21.15.6:0/455610322 learned_addr learned my addr 172.21.15.6:0/455610322 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e659b700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 0x7f28e0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0105250 0x7f28e019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 -- 172.21.15.6:0/455610322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0105250 msgr2=0x7f28e019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0105250 0x7f28e019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 -- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 msgr2=0x7f28e0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 0x7f28e0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 -- 172.21.15.6:0/455610322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc0095f0 con 0x7f28e0068770 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.512+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f28d000e9a0 tx=0x7f28d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.513+0000 7f28d77fe700 1 -- 172.21.15.6:0/455610322 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0009790 con 0x7f28e0068770 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.513+0000 7f28d77fe700 1 -- 172.21.15.6:0/455610322 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d000cbd0 con 0x7f28e0068770 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.513+0000 7f28d77fe700 1 -- 172.21.15.6:0/455610322 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0018680 con 0x7f28e0068770 2023-11-17T00:11:44.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.513+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e0110400 con 0x7f28e0068770 2023-11-17T00:11:44.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.513+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e0110270 con 0x7f28e0068770 2023-11-17T00:11:44.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.517+0000 7f28d77fe700 1 -- 172.21.15.6:0/455610322 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f28d000cd40 con 0x7f28e0068770 2023-11-17T00:11:44.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 Processor -- start 2023-11-17T00:11:44.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 -- start start 2023-11-17T00:11:44.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28d77fe700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 0x7f28cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4cd470 0x56141e4d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28d77fe700 1 -- 172.21.15.6:0/455610322 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28d0014070 con 0x7f28e0068770 2023-11-17T00:11:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e3d2060 con 0x56141e4cd470 2023-11-17T00:11:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e4248a0 con 0x56141e4ccb60 2023-11-17T00:11:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.519+0000 7f28fc138b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e41dd30 con 0x56141e4d18b0 2023-11-17T00:11:44.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 0x7f28cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55788/0 (socket says 172.21.15.6:55788) 2023-11-17T00:11:44.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 -- 172.21.15.6:0/1695166517 learned_addr learned my addr 172.21.15.6:0/1695166517 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:44.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 -- 172.21.15.6:0/1695166517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 msgr2=0x56141e4d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 -- 172.21.15.6:0/1695166517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4cd470 msgr2=0x56141e4d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4cd470 0x56141e4d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e659b700 1 -- 172.21.15.6:0/1695166517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e29f630 con 0x56141e4ccb60 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.520+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.521+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 0x7f28cc0a37f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f28e0105e80 tx=0x7f28d8009920 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.522+0000 7f28e659b700 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005ea0 tx=0x7f28dc005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66c1266f8a81dc60 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.522+0000 7f28d4ff9700 1 -- 172.21.15.6:0/1695166517 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01d070 con 0x56141e4ccb60 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.522+0000 7f28d4ff9700 1 -- 172.21.15.6:0/1695166517 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28dc01e040 con 0x56141e4ccb60 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.522+0000 7f28d4ff9700 1 -- 172.21.15.6:0/1695166517 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004a40 con 0x56141e4ccb60 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.523+0000 7f28fc138b80 1 -- 172.21.15.6:0/1695166517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 msgr2=0x56141e4ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.523+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005ea0 tx=0x7f28dc005ed0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 -- 172.21.15.6:0/1695166517 shutdown_connections 2023-11-17T00:11:44.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4ccb60 0x56141e4ccf30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1695166517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4cd470 0x56141e4d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 -- 172.21.15.6:0/1695166517 >> 172.21.15.6:0/1695166517 conn(0x56141e4c2360 msgr2=0x56141e4c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 -- 172.21.15.6:0/1695166517 shutdown_connections 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 -- 172.21.15.6:0/1695166517 wait complete. 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.524+0000 7f28fc138b80 1 -- start start 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 Processor -- start 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e4d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e4e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e3bf8e0 con 0x56141e4d18b0 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e3bf580 con 0x56141e4cd470 2023-11-17T00:11:44.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e2a1920 con 0x56141e4d6fc0 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.525+0000 7f28fc138b80 10 client.0 _ll_register_callbacks cb 0x56141e3dd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e5d9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e4d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55794/0 (socket says 172.21.15.6:55794) 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 -- 172.21.15.6:0/1494571419 learned_addr learned my addr 172.21.15.6:0/1494571419 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e4e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 -- 172.21.15.6:0/1494571419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 msgr2=0x56141e4e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e4e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 -- 172.21.15.6:0/1494571419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 msgr2=0x56141e4d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e4d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e659b700 1 -- 172.21.15.6:0/1494571419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e010da60 con 0x56141e4cd470 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.526+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e4e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:44.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28e659b700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28dc000c00 tx=0x7f28dc0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cede22b37ae7d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01d040 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56141e4bfb20 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28dc004840 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28dc005bc0 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28fc138b80 10 client.26183 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.527+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56141e3e4190 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56141e4bfb20 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28dc00fc30 con 0x56141e4cd470 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 10 client.26183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 20 client.26183 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:44.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e4e6910 0x56141e4e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 4 client.26183 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56141e3e00a0 con 0x56141e4e6910 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.528+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e4e6910 0x56141e4e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.529+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e4e6910 0x56141e4e8cf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4f8ef83b2bff31 server_cookie=91dab680149ea227 in_seq=0 out_seq=0 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.529+0000 7f28c9ffb700 10 client.26183 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.529+0000 7f28c9ffb700 1 -- 172.21.15.6:0/1494571419 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56141e3e00a0 con 0x56141e4e6910 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.529+0000 7f28c9ffb700 10 client.26183 handle_command_reply: tid=0 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.530+0000 7f28fc138b80 1 client.26183 shutdown 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.530+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 msgr2=0x56141e4d6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.530+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28dc000c00 tx=0x7f28dc0058e0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 shutdown_connections 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e4e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4d6540 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e4e6910 0x56141e4e8cf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 --2- 172.21.15.6:0/1494571419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e4d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 >> 172.21.15.6:0/1494571419 conn(0x56141e4c2360 msgr2=0x56141e4d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28c9ffb700 0 client.26183 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 shutdown_connections 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 1 -- 172.21.15.6:0/1494571419 wait complete. 2023-11-17T00:11:44.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.531+0000 7f28fc138b80 20 client.26183 trim_cache size 0 max 16384 2023-11-17T00:11:44.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 Processor -- start 2023-11-17T00:11:44.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 -- start start 2023-11-17T00:11:44.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4dd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 0x56141e4d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e4d9360 con 0x56141e4d6fc0 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e4cd970 con 0x56141e4cd470 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.549+0000 7f28fc138b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e4d4aa0 con 0x56141e4d18b0 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.550+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.550+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55804/0 (socket says 172.21.15.6:55804) 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.550+0000 7f28e659b700 1 -- 172.21.15.6:0/3906728485 learned_addr learned my addr 172.21.15.6:0/3906728485 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.550+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 0x56141e4d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.550+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4dd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 -- 172.21.15.6:0/3906728485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 msgr2=0x56141e4cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 -- 172.21.15.6:0/3906728485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 msgr2=0x56141e4d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 0x56141e4d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 -- 172.21.15.6:0/3906728485 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e4e34f0 con 0x56141e4d18b0 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e6d9c700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 0x56141e4d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4dd950 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28d00ca790 tx=0x7f28d001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d62963ef1631199 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28d57fa700 1 -- 172.21.15.6:0/3906728485 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00cd120 con 0x56141e4d18b0 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28d57fa700 1 -- 172.21.15.6:0/3906728485 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d00984b0 con 0x56141e4d18b0 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.551+0000 7f28d57fa700 1 -- 172.21.15.6:0/3906728485 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0099600 con 0x56141e4d18b0 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.552+0000 7f28fc138b80 1 -- 172.21.15.6:0/3906728485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 msgr2=0x56141e4dd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.552+0000 7f28fc138b80 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4dd950 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28d00ca790 tx=0x7f28d001a040 comp rx=0 tx=0).stop 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.552+0000 7f28fc138b80 1 -- 172.21.15.6:0/3906728485 shutdown_connections 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d6fc0 0x56141e4d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e4cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 --2- 172.21.15.6:0/3906728485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d18b0 0x56141e4dd950 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 -- 172.21.15.6:0/3906728485 >> 172.21.15.6:0/3906728485 conn(0x56141e4c2360 msgr2=0x56141e4e8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 -- 172.21.15.6:0/3906728485 shutdown_connections 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 -- 172.21.15.6:0/3906728485 wait complete. 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.553+0000 7f28fc138b80 1 -- start start 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 Processor -- start 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e52eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e52f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e539790 con 0x56141e4d18b0 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e539900 con 0x56141e4cd470 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e539ac0 con 0x56141e4d6fc0 2023-11-17T00:11:44.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28fc138b80 10 client.0 _ll_register_callbacks cb 0x7f28dc009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28e659b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55810/0 (socket says 172.21.15.6:55810) 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.554+0000 7f28e659b700 1 -- 172.21.15.6:0/2417483319 learned_addr learned my addr 172.21.15.6:0/2417483319 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e52eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 -- 172.21.15.6:0/2417483319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 msgr2=0x56141e52f0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e52f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 -- 172.21.15.6:0/2417483319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 msgr2=0x56141e52eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e52eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 -- 172.21.15.6:0/2417483319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e0110270 con 0x56141e4cd470 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e52eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28e659b700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02c530 tx=0x7f28dc00b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733085ad206fe968 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:44.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01d030 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.555+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28dc01e040 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28dc000e40 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28dc005bc0 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 10 client.26189 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56141e3e3ee0 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28dc01e040 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28dc036070 con 0x56141e4cd470 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 10 client.26189 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 20 client.26189 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.556+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e5435c0 0x56141e5459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.557+0000 7f28fc138b80 4 client.26189 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:11:44.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.557+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56141e3ec660 con 0x56141e5435c0 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.557+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e5435c0 0x56141e5459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.557+0000 7f28e5d9a700 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e5435c0 0x56141e5459d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daa69714738f5fc server_cookie=14690711a4fa5625 in_seq=0 out_seq=0 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.557+0000 7f28c9ffb700 10 client.26189 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28c9ffb700 1 -- 172.21.15.6:0/2417483319 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56141e3ec660 con 0x56141e5435c0 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28c9ffb700 10 client.26189 handle_command_reply: tid=0 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 client.26189 shutdown 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 msgr2=0x56141e52e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28dc02c530 tx=0x7f28dc00b320 comp rx=0 tx=0).stop 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 shutdown_connections 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56141e4d6fc0 0x56141e52f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56141e4cd470 0x56141e52e620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56141e5435c0 0x56141e5459d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 --2- 172.21.15.6:0/2417483319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56141e4d18b0 0x56141e52eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.558+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 >> 172.21.15.6:0/2417483319 conn(0x56141e4c2360 msgr2=0x56141e4d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.559+0000 7f28c9ffb700 0 client.26189 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.559+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 shutdown_connections 2023-11-17T00:11:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.559+0000 7f28fc138b80 1 -- 172.21.15.6:0/2417483319 wait complete. 2023-11-17T00:11:44.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.559+0000 7f28fc138b80 20 client.26189 trim_cache size 0 max 16384 2023-11-17T00:11:44.570 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 msgr2=0x7f28cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 0x7f28cc0a37f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f28e0105e80 tx=0x7f28d8009920 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 msgr2=0x7f28e0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f28d000e9a0 tx=0x7f28d000ecb0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 shutdown_connections 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f28cc0a1340 0x7f28cc0a37f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28e0067dc0 0x7f28e0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0105250 0x7f28e019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 --2- 172.21.15.6:0/455610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28e0068770 0x7f28e0198b90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:44.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.560+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 >> 172.21.15.6:0/455610322 conn(0x7f28e00763a0 msgr2=0x7f28e010b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:44.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.561+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 shutdown_connections 2023-11-17T00:11:44.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:44.561+0000 7f28e87ff700 1 -- 172.21.15.6:0/455610322 wait complete. 2023-11-17T00:11:44.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:44 smithi006 ceph-mon[97840]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:44.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:44 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3992074786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:45.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:44 smithi087 ceph-mon[131182]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:45.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:44 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3992074786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:45.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:44 smithi186 ceph-mon[131487]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 284 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 MiB/s wr, 1.20k op/s 2023-11-17T00:11:45.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:44 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3992074786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:11:45.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:11:45.589 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 -- 172.21.15.6:0/3104649320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28074400 msgr2=0x7f0e280747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3104649320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28074400 0x7f0e280747d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009990 tx=0x7f0e1c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 -- 172.21.15.6:0/3104649320 shutdown_connections 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3104649320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e280736d0 0x7f0e28113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3104649320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28074400 0x7f0e280747d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3104649320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e28072d20 0x7f0e28073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.871+0000 7f0e2edff700 1 -- 172.21.15.6:0/3104649320 >> 172.21.15.6:0/3104649320 conn(0x7f0e28077500 msgr2=0x7f0e28077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.872+0000 7f0e2edff700 1 -- 172.21.15.6:0/3104649320 shutdown_connections 2023-11-17T00:11:45.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.872+0000 7f0e2edff700 1 -- 172.21.15.6:0/3104649320 wait complete. 2023-11-17T00:11:45.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.873+0000 7f0e2edff700 1 Processor -- start 2023-11-17T00:11:45.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.873+0000 7f0e2edff700 1 -- start start 2023-11-17T00:11:45.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e280736d0 0x7f0e281a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 0x7f0e2819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2819cd60 con 0x7f0e280736d0 2023-11-17T00:11:45.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2819ced0 con 0x7f0e28074400 2023-11-17T00:11:45.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2edff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2819d090 con 0x7f0e28072d20 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2cb9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e27fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e280736d0 0x7f0e281a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2d39c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 0x7f0e2819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2cb9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51350/0 (socket says 172.21.15.6:51350) 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.874+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3118046532 learned_addr learned my addr 172.21.15.6:0/3118046532 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3118046532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 msgr2=0x7f0e2819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 0x7f0e2819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e280736d0 msgr2=0x7f0e281a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e280736d0 0x7f0e281a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3118046532 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0095f0 con 0x7f0e28072d20 2023-11-17T00:11:45.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 0x7f0e2819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:45.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fb0 tx=0x7f0e1c005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e25ffb700 1 -- 172.21.15.6:0/3118046532 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00f990 con 0x7f0e28072d20 2023-11-17T00:11:45.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e25ffb700 1 -- 172.21.15.6:0/3118046532 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e1c02e070 con 0x7f0e28072d20 2023-11-17T00:11:45.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e28116990 con 0x7f0e28072d20 2023-11-17T00:11:45.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.875+0000 7f0e25ffb700 1 -- 172.21.15.6:0/3118046532 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0173f0 con 0x7f0e28072d20 2023-11-17T00:11:45.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.876+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e28116800 con 0x7f0e28072d20 2023-11-17T00:11:45.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.881+0000 7f0e25ffb700 1 -- 172.21.15.6:0/3118046532 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0e1c00faf0 con 0x7f0e28072d20 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.882+0000 7f0e25ffb700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 0x7f0e180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.882+0000 7f0e25ffb700 1 -- 172.21.15.6:0/3118046532 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e1c0df300 con 0x7f0e28072d20 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.882+0000 7f0e27fff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 0x7f0e180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 Processor -- start 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 -- start start 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e27fff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 0x7f0e180a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0e2819db60 tx=0x7f0e1400a380 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160697cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697d4f0 0x5616069813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616068820e0 con 0x56160697cbe0 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616068d4920 con 0x56160697d4f0 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.883+0000 7f0e4268db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616068cddb0 con 0x561606981930 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51356/0 (socket says 172.21.15.6:51356) 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 -- 172.21.15.6:0/525289942 learned_addr learned my addr 172.21.15.6:0/525289942 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e27fff700 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697d4f0 0x5616069813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160697cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 -- 172.21.15.6:0/525289942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697d4f0 msgr2=0x5616069813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697d4f0 0x5616069813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 -- 172.21.15.6:0/525289942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 msgr2=0x56160697cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160697cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 -- 172.21.15.6:0/525289942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160674f6b0 con 0x561606981930 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.884+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007680 tx=0x7f0e20007990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e8f783a0013748 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.885+0000 7f0e12ffd700 1 -- 172.21.15.6:0/525289942 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20003e10 con 0x561606981930 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.885+0000 7f0e12ffd700 1 -- 172.21.15.6:0/525289942 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e2000f040 con 0x561606981930 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.885+0000 7f0e12ffd700 1 -- 172.21.15.6:0/525289942 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20015960 con 0x561606981930 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 -- 172.21.15.6:0/525289942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 msgr2=0x561606981d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007680 tx=0x7f0e20007990 comp rx=0 tx=0).stop 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 -- 172.21.15.6:0/525289942 shutdown_connections 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160697cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x561606981d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 --2- 172.21.15.6:0/525289942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697d4f0 0x5616069813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.886+0000 7f0e4268db80 1 -- 172.21.15.6:0/525289942 >> 172.21.15.6:0/525289942 conn(0x5616069723e0 msgr2=0x561606977bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.887+0000 7f0e4268db80 1 -- 172.21.15.6:0/525289942 shutdown_connections 2023-11-17T00:11:45.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.887+0000 7f0e4268db80 1 -- 172.21.15.6:0/525289942 wait complete. 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.887+0000 7f0e4268db80 1 -- start start 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 Processor -- start 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x561606989130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x561606986340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x561606986a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160686f960 con 0x56160697cbe0 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160686f600 con 0x561606981930 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.888+0000 7f0e4268db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616067519a0 con 0x56160697d4f0 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e4268db80 10 client.0 _ll_register_callbacks cb 0x56160688d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:45.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e27fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x561606986340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e27fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x561606986340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51360/0 (socket says 172.21.15.6:51360) 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e27fff700 1 -- 172.21.15.6:0/150908868 learned_addr learned my addr 172.21.15.6:0/150908868 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x561606986a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x561606989130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 -- 172.21.15.6:0/150908868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 msgr2=0x561606986340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x561606986340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 -- 172.21.15.6:0/150908868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 msgr2=0x561606989130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x561606989130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.889+0000 7f0e2d39c700 1 -- 172.21.15.6:0/150908868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e20009db0 con 0x561606981930 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.890+0000 7f0e2d39c700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x561606986a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e200154c0 tx=0x7f0e20015ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb29ba98948f5a77 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.890+0000 7f0e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.890+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200048d0 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.890+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e20003c10 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.890+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56160696fba0 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.891+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e20003e10 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.891+0000 7f0e4268db80 10 client.26201 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.891+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561606894210 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.891+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e20030030 con 0x561606981930 2023-11-17T00:11:45.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e20005390 con 0x561606981930 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 10 client.26201 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 20 client.26201 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561606996860 0x561606998c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 4 client.26201 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.892+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561606890120 con 0x561606996860 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.893+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561606996860 0x561606998c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.893+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561606996860 0x561606998c70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae48a19d0bb4ce58 server_cookie=7b112543d9085823 in_seq=0 out_seq=0 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.893+0000 7f0e07fff700 10 client.26201 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.894+0000 7f0e07fff700 1 -- 172.21.15.6:0/150908868 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561606890120 con 0x561606996860 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.894+0000 7f0e07fff700 10 client.26201 handle_command_reply: tid=0 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.894+0000 7f0e4268db80 1 client.26201 shutdown 2023-11-17T00:11:45.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 msgr2=0x561606986a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x561606986a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e200154c0 tx=0x7f0e20015ce0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 shutdown_connections 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561606996860 0x561606998c70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x561606989130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x561606986a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 --2- 172.21.15.6:0/150908868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x561606986340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 >> 172.21.15.6:0/150908868 conn(0x5616069723e0 msgr2=0x561606982de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e07fff700 0 client.26201 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 shutdown_connections 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 1 -- 172.21.15.6:0/150908868 wait complete. 2023-11-17T00:11:45.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.895+0000 7f0e4268db80 20 client.26201 trim_cache size 0 max 16384 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.920+0000 7f0e4268db80 1 Processor -- start 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 -- start start 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697d4f0 0x5616069891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x5616069775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616069846d0 con 0x56160697d4f0 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561606971f60 con 0x56160697cbe0 2023-11-17T00:11:45.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e4268db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616069802e0 con 0x561606981930 2023-11-17T00:11:45.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e2cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e2cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55870/0 (socket says 172.21.15.6:55870) 2023-11-17T00:11:45.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3617078262 learned_addr learned my addr 172.21.15.6:0/3617078262 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.921+0000 7f0e27fff700 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697d4f0 0x5616069891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3617078262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 msgr2=0x5616069775e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x5616069775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3617078262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697d4f0 msgr2=0x5616069891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697d4f0 0x5616069891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 -- 172.21.15.6:0/3617078262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561606985eb0 con 0x56160697cbe0 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.922+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00ffd0 tx=0x7f0e1c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b14aee06978d26b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.923+0000 7f0e137fe700 1 -- 172.21.15.6:0/3617078262 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0b4470 con 0x56160697cbe0 2023-11-17T00:11:45.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.923+0000 7f0e137fe700 1 -- 172.21.15.6:0/3617078262 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e1c0aa1f0 con 0x56160697cbe0 2023-11-17T00:11:45.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.923+0000 7f0e137fe700 1 -- 172.21.15.6:0/3617078262 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0ab800 con 0x56160697cbe0 2023-11-17T00:11:45.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.923+0000 7f0e4268db80 1 -- 172.21.15.6:0/3617078262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 msgr2=0x561606988c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.923+0000 7f0e4268db80 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00ffd0 tx=0x7f0e1c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 -- 172.21.15.6:0/3617078262 shutdown_connections 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561606981930 0x5616069775e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56160697cbe0 0x561606988c60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 --2- 172.21.15.6:0/3617078262 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697d4f0 0x5616069891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 -- 172.21.15.6:0/3617078262 >> 172.21.15.6:0/3617078262 conn(0x5616069723e0 msgr2=0x561606992ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 -- 172.21.15.6:0/3617078262 shutdown_connections 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.924+0000 7f0e4268db80 1 -- 172.21.15.6:0/3617078262 wait complete. 2023-11-17T00:11:45.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.925+0000 7f0e4268db80 1 -- start start 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.925+0000 7f0e4268db80 1 Processor -- start 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160698c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x56160698c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x56160698cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616069ea780 con 0x56160697cbe0 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616069ea8c0 con 0x561606981930 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616069eaa80 con 0x56160697d4f0 2023-11-17T00:11:45.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e4268db80 10 client.0 _ll_register_callbacks cb 0x56160688d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:45.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e2cb9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160698c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e2d39c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x56160698cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e2d39c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x56160698cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55886/0 (socket says 172.21.15.6:55886) 2023-11-17T00:11:45.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e2d39c700 1 -- 172.21.15.6:0/1483071755 learned_addr learned my addr 172.21.15.6:0/1483071755 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:45.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.926+0000 7f0e27fff700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x56160698c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 -- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 msgr2=0x56160698cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x56160698cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 -- 172.21.15.6:0/1483071755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 msgr2=0x56160698c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160698c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 -- 172.21.15.6:0/1483071755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e28116800 con 0x56160697d4f0 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e27fff700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x56160698c730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561606978b20 tx=0x7f0e14012070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9acfd8491efcbf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:45.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.927+0000 7f0e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14023040 con 0x56160697d4f0 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e2000f040 con 0x56160697d4f0 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e14010840 con 0x56160697d4f0 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e14010a00 con 0x56160697d4f0 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e4268db80 10 client.26167 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:45.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.928+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5616066fbe60 con 0x56160697d4f0 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e2000f040 con 0x56160697d4f0 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e14028ca0 con 0x56160697d4f0 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 10 client.26167 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 20 client.26167 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5616069ec270 0x5616069ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 4 client.26167 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:11:45.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.929+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56160689c6e0 con 0x5616069ec270 2023-11-17T00:11:45.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.930+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5616069ec270 0x5616069ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:45.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.930+0000 7f0e2cb9b700 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5616069ec270 0x5616069ee680 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b855e8d5befd9a22 server_cookie=88f6eb6671d7d72a in_seq=0 out_seq=0 2023-11-17T00:11:45.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.930+0000 7f0e07fff700 10 client.26167 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:45.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.932+0000 7f0e07fff700 1 -- 172.21.15.6:0/1483071755 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56160689c6e0 con 0x5616069ec270 2023-11-17T00:11:45.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.932+0000 7f0e07fff700 10 client.26167 handle_command_reply: tid=0 2023-11-17T00:11:45.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.932+0000 7f0e4268db80 1 client.26167 shutdown 2023-11-17T00:11:45.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 msgr2=0x56160698c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x56160698c730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561606978b20 tx=0x7f0e14012070 comp rx=0 tx=0).stop 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 shutdown_connections 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5616069ec270 0x5616069ee680 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56160697cbe0 0x56160698c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e07fff700 0 client.26167 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561606981930 0x56160698cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 --2- 172.21.15.6:0/1483071755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56160697d4f0 0x56160698c730 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561606978b20 tx=0x7f0e14012070 comp rx=0 tx=0).stop 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.933+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 >> 172.21.15.6:0/1483071755 conn(0x5616069723e0 msgr2=0x56160697b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.934+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 shutdown_connections 2023-11-17T00:11:45.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.934+0000 7f0e4268db80 1 -- 172.21.15.6:0/1483071755 wait complete. 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.934+0000 7f0e4268db80 20 client.26167 trim_cache size 0 max 16384 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "92abc497-addd-4efc-ae92-66cf79369617", 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:11:45.936 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 msgr2=0x7f0e180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 0x7f0e180a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0e2819db60 tx=0x7f0e1400a380 comp rx=0 tx=0).stop 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 msgr2=0x7f0e281a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fb0 tx=0x7f0e1c005e10 comp rx=0 tx=0).stop 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 shutdown_connections 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0e180a1340 0x7f0e180a37f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e28074400 0x7f0e2819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e28072d20 0x7f0e281a2680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 --2- 172.21.15.6:0/3118046532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e280736d0 0x7f0e281a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:45.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 >> 172.21.15.6:0/3118046532 conn(0x7f0e28077500 msgr2=0x7f0e28112790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:45.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 shutdown_connections 2023-11-17T00:11:45.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:45.935+0000 7f0e2edff700 1 -- 172.21.15.6:0/3118046532 wait complete. 2023-11-17T00:11:45.956 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:11:46.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 -- 172.21.15.6:0/2517931119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c1015f0 msgr2=0x7f4d0c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 --2- 172.21.15.6:0/2517931119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c104e80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009990 tx=0x7f4cfc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 -- 172.21.15.6:0/2517931119 shutdown_connections 2023-11-17T00:11:46.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 --2- 172.21.15.6:0/2517931119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c100c50 0x7f4d0c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 --2- 172.21.15.6:0/2517931119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c105560 0x7f4d0c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.223+0000 7f4d139ff700 1 --2- 172.21.15.6:0/2517931119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c104e80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.224+0000 7f4d139ff700 1 -- 172.21.15.6:0/2517931119 >> 172.21.15.6:0/2517931119 conn(0x7f4d0c077500 msgr2=0x7f4d0c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.225+0000 7f4d139ff700 1 -- 172.21.15.6:0/2517931119 shutdown_connections 2023-11-17T00:11:46.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.225+0000 7f4d139ff700 1 -- 172.21.15.6:0/2517931119 wait complete. 2023-11-17T00:11:46.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.226+0000 7f4d139ff700 1 Processor -- start 2023-11-17T00:11:46.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.226+0000 7f4d139ff700 1 -- start start 2023-11-17T00:11:46.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c100c50 0x7f4d0c194100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c194640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1990e0 con 0x7f4d0c105560 2023-11-17T00:11:46.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c199250 con 0x7f4d0c1015f0 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d139ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c199410 con 0x7f4d0c100c50 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d11f9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d11f9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39224/0 (socket says 172.21.15.6:39224) 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.227+0000 7f4d11f9c700 1 -- 172.21.15.6:0/14458472 learned_addr learned my addr 172.21.15.6:0/14458472 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d1179b700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c100c50 0x7f4d0c194100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c194640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 -- 172.21.15.6:0/14458472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c100c50 msgr2=0x7f4d0c194100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c100c50 0x7f4d0c194100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 -- 172.21.15.6:0/14458472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 msgr2=0x7f4d0c194640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c194640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 -- 172.21.15.6:0/14458472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0095f0 con 0x7f4d0c105560 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.228+0000 7f4d11f9c700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400ef30 tx=0x7f4d0400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.229+0000 7f4d027fc700 1 -- 172.21.15.6:0/14458472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04018070 con 0x7f4d0c105560 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.229+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c10e170 con 0x7f4d0c105560 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.229+0000 7f4d027fc700 1 -- 172.21.15.6:0/14458472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d0400f040 con 0x7f4d0c105560 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.229+0000 7f4d027fc700 1 -- 172.21.15.6:0/14458472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04003dc0 con 0x7f4d0c105560 2023-11-17T00:11:46.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.229+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c10dfe0 con 0x7f4d0c105560 2023-11-17T00:11:46.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.235+0000 7f4d27398b80 1 Processor -- start 2023-11-17T00:11:46.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.235+0000 7f4d027fc700 1 -- 172.21.15.6:0/14458472 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4d0400caa0 con 0x7f4d0c105560 2023-11-17T00:11:46.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.235+0000 7f4d27398b80 1 -- start start 2023-11-17T00:11:46.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652154c3060 con 0x5652155be470 2023-11-17T00:11:46.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652155158a0 con 0x5652155bdb60 2023-11-17T00:11:46.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d27398b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56521550ed30 con 0x5652155c28b0 2023-11-17T00:11:46.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d10f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d10f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55908/0 (socket says 172.21.15.6:55908) 2023-11-17T00:11:46.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.236+0000 7f4d10f9a700 1 -- 172.21.15.6:0/3174819075 learned_addr learned my addr 172.21.15.6:0/3174819075 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:46.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 -- 172.21.15.6:0/3174819075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 msgr2=0x5652155c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 -- 172.21.15.6:0/3174819075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 msgr2=0x5652155c2370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155c2370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 -- 172.21.15.6:0/3174819075 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565215390630 con 0x5652155bdb60 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.237+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c1019c0 tx=0x7f4cfc00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7072cb223f231b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.238+0000 7f4d027fc700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 0x7f4cf80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.238+0000 7f4cf77fe700 1 -- 172.21.15.6:0/3174819075 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc005320 con 0x5652155bdb60 2023-11-17T00:11:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.238+0000 7f4cf77fe700 1 -- 172.21.15.6:0/3174819075 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cfc005480 con 0x5652155bdb60 2023-11-17T00:11:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.238+0000 7f4cf77fe700 1 -- 172.21.15.6:0/3174819075 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00f460 con 0x5652155bdb60 2023-11-17T00:11:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.238+0000 7f4d027fc700 1 -- 172.21.15.6:0/14458472 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4d040ccc70 con 0x7f4d0c105560 2023-11-17T00:11:46.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.239+0000 7f4d1179b700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 0x7f4cf80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.239+0000 7f4d27398b80 1 -- 172.21.15.6:0/3174819075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 msgr2=0x5652155bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c1019c0 tx=0x7f4cfc00b610 comp rx=0 tx=0).stop 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 -- 172.21.15.6:0/3174819075 shutdown_connections 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155bdb60 0x5652155bdf30 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c1019c0 tx=0x7f4cfc00b610 comp rx=0 tx=0).stop 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 --2- 172.21.15.6:0/3174819075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 -- 172.21.15.6:0/3174819075 >> 172.21.15.6:0/3174819075 conn(0x5652155b3360 msgr2=0x5652155b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 -- 172.21.15.6:0/3174819075 shutdown_connections 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d27398b80 1 -- 172.21.15.6:0/3174819075 wait complete. 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.240+0000 7f4d1179b700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 0x7f4cf80a37e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005fd0 tx=0x7f4d08005e30 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.241+0000 7f4d27398b80 1 -- start start 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.241+0000 7f4d27398b80 1 Processor -- start 2023-11-17T00:11:46.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155be470 0x5652155c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x5652155d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652154b08e0 con 0x5652155c7f80 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652154b0580 con 0x5652155c28b0 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x565215392920 con 0x5652155be470 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.242+0000 7f4d27398b80 10 client.0 _ll_register_callbacks cb 0x5652154ce780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55920/0 (socket says 172.21.15.6:55920) 2023-11-17T00:11:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 -- 172.21.15.6:0/894027662 learned_addr learned my addr 172.21.15.6:0/894027662 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 -- 172.21.15.6:0/894027662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155be470 msgr2=0x5652155c7500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155be470 0x5652155c7500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.243+0000 7f4d10f9a700 1 -- 172.21.15.6:0/894027662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 msgr2=0x5652155d3990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4d11f9c700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x5652155d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x5652155d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4d10f9a700 1 -- 172.21.15.6:0/894027662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c10d940 con 0x5652155c28b0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4d11f9c700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x5652155d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc01e040 tx=0x7f4cfc00ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822958e8c874fdc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.244+0000 7f4cf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.245+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc004910 con 0x5652155c28b0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.245+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652155b0b20 con 0x5652155c28b0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.245+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4cfc00f460 con 0x5652155c28b0 2023-11-17T00:11:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.246+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cfc004a70 con 0x5652155c28b0 2023-11-17T00:11:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.246+0000 7f4d27398b80 10 client.26219 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.246+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5652154d5190 con 0x5652155c28b0 2023-11-17T00:11:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.246+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652155b0b20 con 0x5652155c28b0 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.246+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4cfc0296b0 con 0x5652155c28b0 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.247+0000 7f4d27398b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.247+0000 7f4d27398b80 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.247+0000 7f4d27398b80 20 client.26219 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.248+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5652155d78f0 0x5652155d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.248+0000 7f4d27398b80 4 client.26219 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.248+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652154d10a0 con 0x5652155d78f0 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.248+0000 7f4d1179b700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5652155d78f0 0x5652155d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.249+0000 7f4cf4ff9700 10 client.26219 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:46.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.249+0000 7f4d1179b700 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5652155d78f0 0x5652155d9cd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dba9e1c7702176 server_cookie=afbc7f5d41ac6afb in_seq=0 out_seq=0 2023-11-17T00:11:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.251+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/894027662 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5652154d10a0 con 0x5652155d78f0 2023-11-17T00:11:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.251+0000 7f4cf4ff9700 10 client.26219 handle_command_reply: tid=0 2023-11-17T00:11:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.251+0000 7f4d27398b80 1 client.26219 shutdown 2023-11-17T00:11:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.251+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 msgr2=0x5652155c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.251+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc01e040 tx=0x7f4cfc00ba40 comp rx=0 tx=0).stop 2023-11-17T00:11:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 shutdown_connections 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5652155d78f0 0x5652155d9cd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x5652155d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155be470 0x5652155c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 --2- 172.21.15.6:0/894027662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c28b0 0x5652155c7a40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 >> 172.21.15.6:0/894027662 conn(0x5652155b3360 msgr2=0x5652155b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4cf4ff9700 0 client.26219 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 shutdown_connections 2023-11-17T00:11:46.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 1 -- 172.21.15.6:0/894027662 wait complete. 2023-11-17T00:11:46.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.252+0000 7f4d27398b80 20 client.26219 trim_cache size 0 max 16384 2023-11-17T00:11:46.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 Processor -- start 2023-11-17T00:11:46.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 -- start start 2023-11-17T00:11:46.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 0x5652155d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652155bebc0 con 0x5652155be470 2023-11-17T00:11:46.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652155ca6a0 con 0x5652155c7f80 2023-11-17T00:11:46.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.279+0000 7f4d27398b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652155d47d0 con 0x5652155c28b0 2023-11-17T00:11:46.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.280+0000 7f4d11f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 0x5652155d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.280+0000 7f4d10f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.280+0000 7f4d11f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 0x5652155d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55928/0 (socket says 172.21.15.6:55928) 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.280+0000 7f4d1179b700 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.280+0000 7f4d11f9c700 1 -- 172.21.15.6:0/39315615 learned_addr learned my addr 172.21.15.6:0/39315615 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 -- 172.21.15.6:0/39315615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 msgr2=0x5652155d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 0x5652155d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 -- 172.21.15.6:0/39315615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 msgr2=0x5652155ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 -- 172.21.15.6:0/39315615 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652155b4b00 con 0x5652155c28b0 2023-11-17T00:11:46.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.281+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155b8560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc004c90 tx=0x7f4cfc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a80e743879a93b68 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.282+0000 7f4cf7fff700 1 -- 172.21.15.6:0/39315615 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc005d00 con 0x5652155c28b0 2023-11-17T00:11:46.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.282+0000 7f4cf7fff700 1 -- 172.21.15.6:0/39315615 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cfc029d30 con 0x5652155c28b0 2023-11-17T00:11:46.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.282+0000 7f4cf7fff700 1 -- 172.21.15.6:0/39315615 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0173f0 con 0x5652155c28b0 2023-11-17T00:11:46.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 -- 172.21.15.6:0/39315615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 msgr2=0x5652155b8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155b8560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc004c90 tx=0x7f4cfc0098c0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 -- 172.21.15.6:0/39315615 shutdown_connections 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155be470 0x5652155ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155c7f80 0x5652155d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 --2- 172.21.15.6:0/39315615 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155b8560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.283+0000 7f4d27398b80 1 -- 172.21.15.6:0/39315615 >> 172.21.15.6:0/39315615 conn(0x5652155b3360 msgr2=0x5652155c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.284+0000 7f4d27398b80 1 -- 172.21.15.6:0/39315615 shutdown_connections 2023-11-17T00:11:46.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.284+0000 7f4d27398b80 1 -- 172.21.15.6:0/39315615 wait complete. 2023-11-17T00:11:46.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.284+0000 7f4d27398b80 1 -- start start 2023-11-17T00:11:46.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.284+0000 7f4d27398b80 1 Processor -- start 2023-11-17T00:11:46.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.285+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.285+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d27398b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x565215622b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d27398b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652156230c0 con 0x5652155c7f80 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d27398b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x565215623230 con 0x5652155be470 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d27398b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652156233f0 con 0x5652155c28b0 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:55932/0 (socket says 172.21.15.6:55932) 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 -- 172.21.15.6:0/2922502716 learned_addr learned my addr 172.21.15.6:0/2922502716 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:46.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d27398b80 10 client.0 _ll_register_callbacks cb 0x7f4d040097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 -- 172.21.15.6:0/2922502716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 msgr2=0x5652155ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 -- 172.21.15.6:0/2922502716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 msgr2=0x565215622b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x565215622b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d1179b700 1 -- 172.21.15.6:0/2922502716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652154d5190 con 0x5652155be470 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.286+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.287+0000 7f4d1179b700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800a480 tx=0x7f4d08014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdede630a39373c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:46.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.287+0000 7f4cf4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:46.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.287+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080235c0 con 0x5652155be470 2023-11-17T00:11:46.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.287+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4d08004830 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cfc005480 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d080049f0 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4d27398b80 10 client.26225 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56521533cde0 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cfc005480 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.288+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4d080104c0 con 0x5652155be470 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 10 client.26225 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 10 client.26225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 20 client.26225 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56521562be60 0x56521562e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 4 client.26225 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652154dd660 con 0x56521562be60 2023-11-17T00:11:46.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.289+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56521562be60 0x56521562e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.290+0000 7f4d10f9a700 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56521562be60 0x56521562e270 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3094ac101f06a1c1 server_cookie=7fd987f94577c602 in_seq=0 out_seq=0 2023-11-17T00:11:46.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.290+0000 7f4cf4ff9700 10 client.26225 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4cf4ff9700 1 -- 172.21.15.6:0/2922502716 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5652154dd660 con 0x56521562be60 2023-11-17T00:11:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4cf4ff9700 10 client.26225 handle_command_reply: tid=0 2023-11-17T00:11:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 client.26225 shutdown 2023-11-17T00:11:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 msgr2=0x5652155cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800a480 tx=0x7f4d08014000 comp rx=0 tx=0).stop 2023-11-17T00:11:46.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 shutdown_connections 2023-11-17T00:11:46.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56521562be60 0x56521562e270 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4cf4ff9700 0 client.26225 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5652155c7f80 0x565215622b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5652155be470 0x5652155cdf40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 --2- 172.21.15.6:0/2922502716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5652155c28b0 0x5652155ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 >> 172.21.15.6:0/2922502716 conn(0x5652155b3360 msgr2=0x5652155d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 shutdown_connections 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.292+0000 7f4d27398b80 1 -- 172.21.15.6:0/2922502716 wait complete. 2023-11-17T00:11:46.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.293+0000 7f4d27398b80 20 client.26225 trim_cache size 0 max 16384 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (3687 inodes in the stack)", 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "92abc497-addd-4efc-ae92-66cf79369617": { 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "92abc497-addd-4efc-ae92-66cf79369617", 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:11:46.296 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:11:46.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.294+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 msgr2=0x7f4cf80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.294+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 0x7f4cf80a37e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005fd0 tx=0x7f4d08005e30 comp rx=0 tx=0).stop 2023-11-17T00:11:46.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.294+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 msgr2=0x7f4d0c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:46.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.294+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400ef30 tx=0x7f4d0400c450 comp rx=0 tx=0).stop 2023-11-17T00:11:46.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 shutdown_connections 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4cf80a1330 0x7f4cf80a37e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d0c105560 0x7f4d0c198aa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d0c100c50 0x7f4d0c194100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 --2- 172.21.15.6:0/14458472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d0c1015f0 0x7f4d0c194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 >> 172.21.15.6:0/14458472 conn(0x7f4d0c077500 msgr2=0x7f4d0c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 shutdown_connections 2023-11-17T00:11:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:11:46.295+0000 7f4d139ff700 1 -- 172.21.15.6:0/14458472 wait complete. 2023-11-17T00:11:46.321 INFO:tasks.cephfs.filesystem:scrub status for tag:92abc497-addd-4efc-ae92-66cf79369617 - {'path': '/', 'tag': '92abc497-addd-4efc-ae92-66cf79369617', 'options': 'recursive,force'} 2023-11-17T00:11:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:46 smithi006 ceph-mon[97840]: pgmap v89: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 3.20k op/s 2023-11-17T00:11:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:46 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:11:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:46 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:46 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:11:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:46 smithi087 ceph-mon[131182]: pgmap v89: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 3.20k op/s 2023-11-17T00:11:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:46 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:11:47.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:46 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:47.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:46 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:11:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:46 smithi186 ceph-mon[131487]: pgmap v89: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 3.20k op/s 2023-11-17T00:11:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:46 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:11:47.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:46 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:11:47.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:46 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:11:48.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:48 smithi006 ceph-mon[97840]: pgmap v90: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:48 smithi087 ceph-mon[131182]: pgmap v90: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:48 smithi186 ceph-mon[131487]: pgmap v90: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:50 smithi006 ceph-mon[97840]: pgmap v91: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:50 smithi087 ceph-mon[131182]: pgmap v91: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:50 smithi186 ceph-mon[131487]: pgmap v91: 129 pgs: 129 active+clean; 8.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 3.20k op/s 2023-11-17T00:11:52.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:52 smithi006 ceph-mon[97840]: pgmap v92: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 3.56k op/s 2023-11-17T00:11:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:52 smithi087 ceph-mon[131182]: pgmap v92: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 3.56k op/s 2023-11-17T00:11:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:52 smithi186 ceph-mon[131487]: pgmap v92: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 3.56k op/s 2023-11-17T00:11:54.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:54 smithi006 ceph-mon[97840]: pgmap v93: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 2.37k op/s 2023-11-17T00:11:56.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:54 smithi186 ceph-mon[131487]: pgmap v93: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 2.37k op/s 2023-11-17T00:11:56.616 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:54 smithi087 ceph-mon[131182]: pgmap v93: 129 pgs: 129 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 2.37k op/s 2023-11-17T00:11:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 -- 172.21.15.6:0/226786583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075020 msgr2=0x7f4c3c0753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 --2- 172.21.15.6:0/226786583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075020 0x7f4c3c0753f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c380098f0 tx=0x7f4c38009c00 comp rx=0 tx=0).stop 2023-11-17T00:11:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 -- 172.21.15.6:0/226786583 shutdown_connections 2023-11-17T00:11:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 --2- 172.21.15.6:0/226786583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c111120 0x7f4c3c113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 --2- 172.21.15.6:0/226786583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075020 0x7f4c3c0753f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 --2- 172.21.15.6:0/226786583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c075930 0x7f4c3c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.752+0000 7f4c437ff700 1 -- 172.21.15.6:0/226786583 >> 172.21.15.6:0/226786583 conn(0x7f4c3c0fdf60 msgr2=0x7f4c3c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.753+0000 7f4c437ff700 1 -- 172.21.15.6:0/226786583 shutdown_connections 2023-11-17T00:11:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.753+0000 7f4c437ff700 1 -- 172.21.15.6:0/226786583 wait complete. 2023-11-17T00:11:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.754+0000 7f4c437ff700 1 Processor -- start 2023-11-17T00:11:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.754+0000 7f4c437ff700 1 -- start start 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c075020 0x7f4c3c1a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c111120 0x7f4c3c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c19cdf0 con 0x7f4c3c111120 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c19cf60 con 0x7f4c3c075020 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c437ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c19d120 con 0x7f4c3c075930 2023-11-17T00:11:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c4159b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c075020 0x7f4c3c1a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:35954/0 (socket says 172.21.15.6:35954) 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1156959266 learned_addr learned my addr 172.21.15.6:0/1156959266 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.755+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c111120 0x7f4c3c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1156959266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c075020 msgr2=0x7f4c3c1a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c075020 0x7f4c3c1a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c111120 msgr2=0x7f4c3c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c111120 0x7f4c3c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1156959266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c38009520 con 0x7f4c3c075930 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c9b0 tx=0x7f4c2c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c327fc700 1 -- 172.21.15.6:0/1156959266 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c007b00 con 0x7f4c3c075930 2023-11-17T00:11:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c327fc700 1 -- 172.21.15.6:0/1156959266 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2c00ce50 con 0x7f4c3c075930 2023-11-17T00:11:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.756+0000 7f4c327fc700 1 -- 172.21.15.6:0/1156959266 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c018660 con 0x7f4c3c075930 2023-11-17T00:11:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.757+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c116910 con 0x7f4c3c075930 2023-11-17T00:11:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.757+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c116780 con 0x7f4c3c075930 2023-11-17T00:11:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.762+0000 7f4c327fc700 1 -- 172.21.15.6:0/1156959266 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4c2c00f450 con 0x7f4c3c075930 2023-11-17T00:11:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.763+0000 7f4c57176b80 1 Processor -- start 2023-11-17T00:11:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.763+0000 7f4c327fc700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 0x7f4c280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c327fc700 1 -- 172.21.15.6:0/1156959266 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4c2c0ccfd0 con 0x7f4c3c075930 2023-11-17T00:11:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 0x7f4c280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c57176b80 1 -- start start 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233e941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.764+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c57176b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233d94000 con 0x564233e94b90 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c57176b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233de6840 con 0x564233cea6b0 2023-11-17T00:11:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c57176b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233ddfcd0 con 0x564233e94720 2023-11-17T00:11:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:35964/0 (socket says 172.21.15.6:35964) 2023-11-17T00:11:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1582678525 learned_addr learned my addr 172.21.15.6:0/1582678525 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c4159b700 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233e941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.765+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1582678525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 msgr2=0x564233e941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233e941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1582678525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 msgr2=0x564233e8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1582678525 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564233c615d0 con 0x564233e94720 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.766+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 0x7f4c280a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c1a3410 tx=0x7f4c340099e0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.767+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c096760 tx=0x7f4c2c0996d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfb6bfda8f1a7cf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.767+0000 7f4c277fe700 1 -- 172.21.15.6:0/1582678525 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c09dd20 con 0x564233e94720 2023-11-17T00:11:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.767+0000 7f4c277fe700 1 -- 172.21.15.6:0/1582678525 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2c0a2140 con 0x564233e94720 2023-11-17T00:11:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.767+0000 7f4c277fe700 1 -- 172.21.15.6:0/1582678525 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c099f50 con 0x564233e94720 2023-11-17T00:11:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.768+0000 7f4c57176b80 1 -- 172.21.15.6:0/1582678525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 msgr2=0x564233e8e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.768+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c096760 tx=0x7f4c2c0996d0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 -- 172.21.15.6:0/1582678525 shutdown_connections 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233e941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1582678525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8e560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 -- 172.21.15.6:0/1582678525 >> 172.21.15.6:0/1582678525 conn(0x564233e83f50 msgr2=0x564233e89270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 -- 172.21.15.6:0/1582678525 shutdown_connections 2023-11-17T00:11:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.769+0000 7f4c57176b80 1 -- 172.21.15.6:0/1582678525 wait complete. 2023-11-17T00:11:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.770+0000 7f4c57176b80 1 -- start start 2023-11-17T00:11:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.770+0000 7f4c57176b80 1 Processor -- start 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 0x564233e97d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233d81880 con 0x564233e94720 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233d81520 con 0x564233e94b90 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233c638c0 con 0x564233cea6b0 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c57176b80 10 client.0 _ll_register_callbacks cb 0x564233d9f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c4159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c41d9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c4159b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:35966/0 (socket says 172.21.15.6:35966) 2023-11-17T00:11:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c41d9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:48910/0 (socket says 172.21.15.6:48910) 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.771+0000 7f4c4159b700 1 -- 172.21.15.6:0/553474681 learned_addr learned my addr 172.21.15.6:0/553474681 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 -- 172.21.15.6:0/553474681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 msgr2=0x564233e9aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 -- 172.21.15.6:0/553474681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 msgr2=0x564233e97d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 0x564233e97d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 0x564233e97d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 -- 172.21.15.6:0/553474681 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c116780 con 0x564233e94b90 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c4159b700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 0x564233e97d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564233e94f60 tx=0x7f4c34004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5611cbd9ff5a6c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.772+0000 7f4c24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.773+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e040 con 0x564233e94b90 2023-11-17T00:11:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.773+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4c34004db0 con 0x564233e94b90 2023-11-17T00:11:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.773+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564233e81710 con 0x564233e94b90 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.774+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c3400fe60 con 0x564233e94b90 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.775+0000 7f4c57176b80 10 client.26237 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.775+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564233da6130 con 0x564233e94b90 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.775+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564233e81710 con 0x564233e94b90 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c340041c0 con 0x564233e94b90 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 20 client.26237 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233eb9100 0x564233ebb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 4 client.26237 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:11:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564233da2040 con 0x564233eb9100 2023-11-17T00:11:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.776+0000 7f4c4159b700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233eb9100 0x564233ebb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.777+0000 7f4c4159b700 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233eb9100 0x564233ebb510 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e42d4c3f25a703 server_cookie=9c80303a55be4c7c in_seq=0 out_seq=0 2023-11-17T00:11:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.777+0000 7f4c24ff9700 10 client.26237 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.779+0000 7f4c24ff9700 1 -- 172.21.15.6:0/553474681 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564233da2040 con 0x564233eb9100 2023-11-17T00:11:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.779+0000 7f4c24ff9700 10 client.26237 handle_command_reply: tid=0 2023-11-17T00:11:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.779+0000 7f4c57176b80 1 client.26237 shutdown 2023-11-17T00:11:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.779+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 msgr2=0x564233e98450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.779+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564233e94f60 tx=0x7f4c34004b40 comp rx=0 tx=0).stop 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 shutdown_connections 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e98450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233cea6b0 0x564233e9aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233eb9100 0x564233ebb510 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 --2- 172.21.15.6:0/553474681 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94720 0x564233e97d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 >> 172.21.15.6:0/553474681 conn(0x564233e83f50 msgr2=0x564233e887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c24ff9700 0 client.26237 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 shutdown_connections 2023-11-17T00:11:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 1 -- 172.21.15.6:0/553474681 wait complete. 2023-11-17T00:11:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.780+0000 7f4c57176b80 20 client.26237 trim_cache size 0 max 16384 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.803+0000 7f4c57176b80 1 Processor -- start 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.803+0000 7f4c57176b80 1 -- start start 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 0x564233eb00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e91ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e8f370 con 0x564233cea6b0 2023-11-17T00:11:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e8b050 con 0x564233e94b90 2023-11-17T00:11:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c57176b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e92a20 con 0x564233e94720 2023-11-17T00:11:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c4159b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 0x564233eb00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:35968/0 (socket says 172.21.15.6:35968) 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1854030729 learned_addr learned my addr 172.21.15.6:0/1854030729 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.804+0000 7f4c41d9c700 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e91ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1854030729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 msgr2=0x564233e91ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e91ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1854030729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 msgr2=0x564233eb00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 0x564233eb00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1854030729 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564233da5e80 con 0x564233e94720 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c4159b700 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 0x564233eb00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:11:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.805+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0074a0 tx=0x7f4c2c0960e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3d16e1005e20b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.806+0000 7f4c27fff700 1 -- 172.21.15.6:0/1854030729 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c098090 con 0x564233e94720 2023-11-17T00:11:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.806+0000 7f4c27fff700 1 -- 172.21.15.6:0/1854030729 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2c0ca180 con 0x564233e94720 2023-11-17T00:11:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.806+0000 7f4c27fff700 1 -- 172.21.15.6:0/1854030729 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0991e0 con 0x564233e94720 2023-11-17T00:11:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.806+0000 7f4c57176b80 1 -- 172.21.15.6:0/1854030729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 msgr2=0x564233e88f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.806+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0074a0 tx=0x7f4c2c0960e0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 -- 172.21.15.6:0/1854030729 shutdown_connections 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233cea6b0 0x564233eb00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233e94b90 0x564233e91ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1854030729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e88f60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 -- 172.21.15.6:0/1854030729 >> 172.21.15.6:0/1854030729 conn(0x564233e83f50 msgr2=0x564233eb4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 -- 172.21.15.6:0/1854030729 shutdown_connections 2023-11-17T00:11:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.807+0000 7f4c57176b80 1 -- 172.21.15.6:0/1854030729 wait complete. 2023-11-17T00:11:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.808+0000 7f4c57176b80 1 -- start start 2023-11-17T00:11:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.808+0000 7f4c57176b80 1 Processor -- start 2023-11-17T00:11:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.809+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233eb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.809+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.809+0000 7f4c57176b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:57.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:56 smithi186 ceph-mon[131487]: pgmap v94: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 2.83k op/s 2023-11-17T00:11:57.121 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:56 smithi087 ceph-mon[131182]: pgmap v94: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 2.83k op/s 2023-11-17T00:11:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.809+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.809+0000 7f4c40d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:35972/0 (socket says 172.21.15.6:35972) 2023-11-17T00:11:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c57176b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e906e0 con 0x564233e94b90 2023-11-17T00:11:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c57176b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e90850 con 0x564233cea6b0 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c57176b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564233e90a10 con 0x564233e94720 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c57176b80 10 client.0 _ll_register_callbacks cb 0x7f4c34019ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1791702925 learned_addr learned my addr 172.21.15.6:0/1791702925 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 msgr2=0x564233eb00b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233eb00b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1791702925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 msgr2=0x564233e901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 -- 172.21.15.6:0/1791702925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564233c615d0 con 0x564233e94720 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.810+0000 7f4c40d9a700 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0961f0 tx=0x7f4c2c0996d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1de2b4c9a89614 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.811+0000 7f4c24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.811+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c07d070 con 0x564233e94720 2023-11-17T00:11:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.811+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4c2c098150 con 0x564233e94720 2023-11-17T00:11:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.811+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c2c0a2140 con 0x564233e94720 2023-11-17T00:11:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c2c083080 con 0x564233e94720 2023-11-17T00:11:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 10 client.26200 fetch_fsmap learned FSMap version 22 2023-11-17T00:11:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564233bf6c00 con 0x564233e94720 2023-11-17T00:11:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2c0a2140 con 0x564233e94720 2023-11-17T00:11:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c2c080030 con 0x564233e94720 2023-11-17T00:11:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:11:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 20 client.26200 populate_metadata read hostname 'smithi006' 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.812+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233f0cfc0 0x564233f0f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.813+0000 7f4c57176b80 4 client.26200 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.813+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564233dae600 con 0x564233f0cfc0 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.813+0000 7f4c4159b700 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233f0cfc0 0x564233f0f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.814+0000 7f4c24ff9700 10 client.26200 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:11:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:11:56.814+0000 7f4c4159b700 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233f0cfc0 0x564233f0f3a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19541ed45e18028f server_cookie=fba543504f3b7b08 in_seq=0 out_seq=0 2023-11-17T00:11:57.485 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:56 smithi006 ceph-mon[97840]: pgmap v94: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 2.83k op/s 2023-11-17T00:11:58.982 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:11:58 smithi006 ceph-mon[97840]: pgmap v95: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:11:59.539 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:11:58 smithi186 ceph-mon[131487]: pgmap v95: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:11:59.539 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:11:58 smithi087 ceph-mon[131182]: pgmap v95: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:12:01.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:00 smithi087 ceph-mon[131182]: pgmap v96: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:12:01.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:00 smithi186 ceph-mon[131487]: pgmap v96: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:12:01.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:00 smithi006 ceph-mon[97840]: pgmap v96: 129 pgs: 129 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 825 op/s 2023-11-17T00:12:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:02 smithi087 ceph-mon[131182]: pgmap v97: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.97k op/s 2023-11-17T00:12:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:02 smithi186 ceph-mon[131487]: pgmap v97: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.97k op/s 2023-11-17T00:12:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:02 smithi006 ceph-mon[97840]: pgmap v97: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.97k op/s 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.872+0000 7f4c24ff9700 1 -- 172.21.15.6:0/1791702925 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564233dae600 con 0x564233f0cfc0 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.872+0000 7f4c24ff9700 10 client.26200 handle_command_reply: tid=0 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 client.26200 shutdown 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 msgr2=0x564233e8fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0961f0 tx=0x7f4c2c0996d0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 shutdown_connections 2023-11-17T00:12:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x564233f0cfc0 0x564233f0f3a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x564233e94b90 0x564233e901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564233cea6b0 0x564233eb00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 --2- 172.21.15.6:0/1791702925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564233e94720 0x564233e8fc60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c24ff9700 0 client.26200 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 >> 172.21.15.6:0/1791702925 conn(0x564233e83f50 msgr2=0x564233e91ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 shutdown_connections 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 1 -- 172.21.15.6:0/1791702925 wait complete. 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.873+0000 7f4c57176b80 20 client.26200 trim_cache size 0 max 16384 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:12:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 msgr2=0x7f4c280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 0x7f4c280a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c1a3410 tx=0x7f4c340099e0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 msgr2=0x7f4c3c1a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00c9b0 tx=0x7f4c2c00ccc0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 shutdown_connections 2023-11-17T00:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4c280a12c0 0x7f4c280a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c3c111120 0x7f4c3c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c3c075020 0x7f4c3c1a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 --2- 172.21.15.6:0/1156959266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c3c075930 0x7f4c3c1a2c50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 >> 172.21.15.6:0/1156959266 conn(0x7f4c3c0fdf60 msgr2=0x7f4c3c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 shutdown_connections 2023-11-17T00:12:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:04.874+0000 7f4c437ff700 1 -- 172.21.15.6:0/1156959266 wait complete. 2023-11-17T00:12:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:04 smithi087 ceph-mon[131182]: pgmap v98: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 1.60k op/s 2023-11-17T00:12:05.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:04 smithi186 ceph-mon[131487]: pgmap v98: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 1.60k op/s 2023-11-17T00:12:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:04 smithi006 ceph-mon[97840]: pgmap v98: 129 pgs: 129 active+clean; 34 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 1.60k op/s 2023-11-17T00:12:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:06 smithi087 ceph-mon[131182]: pgmap v99: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 4.05k op/s 2023-11-17T00:12:07.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:06 smithi186 ceph-mon[131487]: pgmap v99: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 4.05k op/s 2023-11-17T00:12:07.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:06 smithi006 ceph-mon[97840]: pgmap v99: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 4.05k op/s 2023-11-17T00:12:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:08 smithi087 ceph-mon[131182]: pgmap v100: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:08 smithi186 ceph-mon[131487]: pgmap v100: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:09.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:08 smithi006 ceph-mon[97840]: pgmap v100: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:10 smithi087 ceph-mon[131182]: pgmap v101: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:10 smithi186 ceph-mon[131487]: pgmap v101: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:11.231 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:10 smithi006 ceph-mon[97840]: pgmap v101: 129 pgs: 129 active+clean; 42 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 3.59k op/s 2023-11-17T00:12:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:12 smithi087 ceph-mon[131182]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 3.77k op/s 2023-11-17T00:12:13.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:12 smithi186 ceph-mon[131487]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 3.77k op/s 2023-11-17T00:12:13.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:12 smithi006 ceph-mon[97840]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 3.77k op/s 2023-11-17T00:12:15.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:14 smithi087 ceph-mon[131182]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 2.63k op/s 2023-11-17T00:12:15.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:14 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:12:15.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:14 smithi186 ceph-mon[131487]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 2.63k op/s 2023-11-17T00:12:15.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:14 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:12:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:14 smithi006 ceph-mon[97840]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 2.63k op/s 2023-11-17T00:12:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:14 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:12:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:15 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:12:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:15 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:12:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:15 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:12:16.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:15 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:12:16.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:15 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:12:16.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:15 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:12:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:15 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:12:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:15 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:12:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:15 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:12:16.321 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:12:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 -- 172.21.15.6:0/1395095346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd41015f0 msgr2=0x7f7dd4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/1395095346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd41015f0 0x7f7dd4104e80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80099c0 tx=0x7f7dc8009cd0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 -- 172.21.15.6:0/1395095346 shutdown_connections 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/1395095346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd4100c50 0x7f7dd4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/1395095346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4105560 0x7f7dd410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/1395095346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd41015f0 0x7f7dd4104e80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 -- 172.21.15.6:0/1395095346 >> 172.21.15.6:0/1395095346 conn(0x7f7dd4077500 msgr2=0x7f7dd4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 -- 172.21.15.6:0/1395095346 shutdown_connections 2023-11-17T00:12:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.537+0000 7f7dd91ff700 1 -- 172.21.15.6:0/1395095346 wait complete. 2023-11-17T00:12:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 Processor -- start 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 -- start start 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 0x7f7dd41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd41015f0 0x7f7dd4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419d5b0 con 0x7f7dd41015f0 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419d720 con 0x7f7dd4105560 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd91ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419d8e0 con 0x7f7dd4100c50 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd41015f0 0x7f7dd4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38112/0 (socket says 172.21.15.6:38112) 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.539+0000 7f7dd359e700 1 -- 172.21.15.6:0/2352488488 learned_addr learned my addr 172.21.15.6:0/2352488488 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 0x7f7dd41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 -- 172.21.15.6:0/2352488488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 msgr2=0x7f7dd41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 0x7f7dd41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 -- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd41015f0 msgr2=0x7f7dd4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd41015f0 0x7f7dd4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 -- 172.21.15.6:0/2352488488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4009520 con 0x7f7dd4105560 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 0x7f7dd41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:12:16.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd359e700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00ef30 tx=0x7f7dcc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dbbfff700 1 -- 172.21.15.6:0/2352488488 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc018070 con 0x7f7dd4105560 2023-11-17T00:12:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dbbfff700 1 -- 172.21.15.6:0/2352488488 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dcc00f040 con 0x7f7dd4105560 2023-11-17T00:12:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc80095f0 con 0x7f7dd4105560 2023-11-17T00:12:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dbbfff700 1 -- 172.21.15.6:0/2352488488 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00cdb0 con 0x7f7dd4105560 2023-11-17T00:12:16.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.540+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4110320 con 0x7f7dd4105560 2023-11-17T00:12:16.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.543+0000 7f7dbbfff700 1 -- 172.21.15.6:0/2352488488 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7dcc003a00 con 0x7f7dd4105560 2023-11-17T00:12:16.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 Processor -- start 2023-11-17T00:12:16.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 -- start start 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7dbbfff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 0x7f7dbc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7dbbfff700 1 -- 172.21.15.6:0/2352488488 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7dcc0ccc10 con 0x7f7dd4105560 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7831b60 0x562da7831f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7dd259c700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 0x7f7dbc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7832470 0x562da7836370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7737060 con 0x562da7831b60 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da77898a0 con 0x562da78368b0 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.546+0000 7f7decbfab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7782d30 con 0x562da7832470 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7832470 0x562da7836370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38128/0 (socket says 172.21.15.6:38128) 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 -- 172.21.15.6:0/908487531 learned_addr learned my addr 172.21.15.6:0/908487531 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7831b60 0x562da7831f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 -- 172.21.15.6:0/908487531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7832470 msgr2=0x562da7836370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7832470 0x562da7836370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 -- 172.21.15.6:0/908487531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7831b60 msgr2=0x562da7831f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7831b60 0x562da7831f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 -- 172.21.15.6:0/908487531 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da7604630 con 0x562da78368b0 2023-11-17T00:12:16.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd359e700 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0a1850 tx=0x7f7dcc099790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20eadf7e3442b97c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7dd259c700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 0x7f7dbc0a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b580 tx=0x7f7dc8004880 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.547+0000 7f7db97fa700 1 -- 172.21.15.6:0/908487531 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc004ad0 con 0x562da78368b0 2023-11-17T00:12:16.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7db97fa700 1 -- 172.21.15.6:0/908487531 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dcc0999c0 con 0x562da78368b0 2023-11-17T00:12:16.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7db97fa700 1 -- 172.21.15.6:0/908487531 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc099e80 con 0x562da78368b0 2023-11-17T00:12:16.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 -- 172.21.15.6:0/908487531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 msgr2=0x562da7836d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0a1850 tx=0x7f7dcc099790 comp rx=0 tx=0).stop 2023-11-17T00:12:16.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 -- 172.21.15.6:0/908487531 shutdown_connections 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7831b60 0x562da7831f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7836d00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 --2- 172.21.15.6:0/908487531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7832470 0x562da7836370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 -- 172.21.15.6:0/908487531 >> 172.21.15.6:0/908487531 conn(0x562da7827360 msgr2=0x562da782cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 -- 172.21.15.6:0/908487531 shutdown_connections 2023-11-17T00:12:16.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.548+0000 7f7decbfab80 1 -- 172.21.15.6:0/908487531 wait complete. 2023-11-17T00:12:16.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.549+0000 7f7decbfab80 1 -- start start 2023-11-17T00:12:16.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.549+0000 7f7decbfab80 1 Processor -- start 2023-11-17T00:12:16.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da783b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da783b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da783be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da77248e0 con 0x562da7832470 2023-11-17T00:12:16.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7724580 con 0x562da78368b0 2023-11-17T00:12:16.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7606920 con 0x562da7831b60 2023-11-17T00:12:16.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7decbfab80 10 client.0 _ll_register_callbacks cb 0x562da7742780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:16.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da783b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da783be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da783b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42648/0 (socket says 172.21.15.6:42648) 2023-11-17T00:12:16.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd259c700 1 -- 172.21.15.6:0/509695379 learned_addr learned my addr 172.21.15.6:0/509695379 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:16.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da783b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 -- 172.21.15.6:0/509695379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 msgr2=0x562da783b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da783b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 -- 172.21.15.6:0/509695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 msgr2=0x562da783b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da783b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.550+0000 7f7dd359e700 1 -- 172.21.15.6:0/509695379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4110320 con 0x562da78368b0 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7dd359e700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da783be20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562da783e620 tx=0x7f7dcc0998c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d8ebea65709f2e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc004ad0 con 0x562da78368b0 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7dcc09d9f0 con 0x562da78368b0 2023-11-17T00:12:16.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da7824b20 con 0x562da78368b0 2023-11-17T00:12:16.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dcc0a1d80 con 0x562da78368b0 2023-11-17T00:12:16.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 10 client.26252 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:16.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x562da7749190 con 0x562da78368b0 2023-11-17T00:12:16.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562da7824b20 con 0x562da78368b0 2023-11-17T00:12:16.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dcc0957b0 con 0x562da78368b0 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 10 client.26252 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 20 client.26252 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da784b490 0x562da784d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 4 client.26252 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.551+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da77450a0 con 0x562da784b490 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.552+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da784b490 0x562da784d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.552+0000 7f7db6ff5700 10 client.26252 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.552+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da784b490 0x562da784d870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d92dfbdf52a8a3 server_cookie=b8f2e87943302a86 in_seq=0 out_seq=0 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7db6ff5700 1 -- 172.21.15.6:0/509695379 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562da77450a0 con 0x562da784b490 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7db6ff5700 10 client.26252 handle_command_reply: tid=0 2023-11-17T00:12:16.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 client.26252 shutdown 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 msgr2=0x562da783be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da783be20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562da783e620 tx=0x7f7dcc0998c0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 shutdown_connections 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da783be20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da783b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da784b490 0x562da784d870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 --2- 172.21.15.6:0/509695379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da783b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7db6ff5700 0 client.26252 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 >> 172.21.15.6:0/509695379 conn(0x562da7827360 msgr2=0x562da7837d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 shutdown_connections 2023-11-17T00:12:16.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 1 -- 172.21.15.6:0/509695379 wait complete. 2023-11-17T00:12:16.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.555+0000 7f7decbfab80 20 client.26252 trim_cache size 0 max 16384 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.573+0000 7f7decbfab80 1 Processor -- start 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.573+0000 7f7decbfab80 1 -- start start 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7834a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7834fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7832bc0 con 0x562da7832470 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da783e730 con 0x562da78368b0 2023-11-17T00:12:16.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7decbfab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7826ee0 con 0x562da7831b60 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7834a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43088/0 (socket says 172.21.15.6:43088) 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 -- 172.21.15.6:0/628327869 learned_addr learned my addr 172.21.15.6:0/628327869 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd359e700 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7834fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 -- 172.21.15.6:0/628327869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 msgr2=0x562da7834fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7834fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 -- 172.21.15.6:0/628327869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 msgr2=0x562da7834a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7834a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 -- 172.21.15.6:0/628327869 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da783af10 con 0x562da7831b60 2023-11-17T00:12:16.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.574+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562da784e180 tx=0x7f7dc4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5141a51d1478eb54 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7db9ffb700 1 -- 172.21.15.6:0/628327869 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004220 con 0x562da7831b60 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7db9ffb700 1 -- 172.21.15.6:0/628327869 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc4018040 con 0x562da7831b60 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7db9ffb700 1 -- 172.21.15.6:0/628327869 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004830 con 0x562da7831b60 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 -- 172.21.15.6:0/628327869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 msgr2=0x562da78429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562da784e180 tx=0x7f7dc4004750 comp rx=0 tx=0).stop 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 -- 172.21.15.6:0/628327869 shutdown_connections 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7834fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78429a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 --2- 172.21.15.6:0/628327869 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7834a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 -- 172.21.15.6:0/628327869 >> 172.21.15.6:0/628327869 conn(0x562da7827360 msgr2=0x562da782b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 -- 172.21.15.6:0/628327869 shutdown_connections 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.575+0000 7f7decbfab80 1 -- 172.21.15.6:0/628327869 wait complete. 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.576+0000 7f7decbfab80 1 -- start start 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.576+0000 7f7decbfab80 1 Processor -- start 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.576+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.576+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.576+0000 7f7decbfab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7decbfab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da78338d0 con 0x562da7832470 2023-11-17T00:12:16.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7decbfab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7833a40 con 0x562da78368b0 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7decbfab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7833c00 con 0x562da7831b60 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7decbfab80 10 client.0 _ll_register_callbacks cb 0x7f7dc40097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42658/0 (socket says 172.21.15.6:42658) 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38148/0 (socket says 172.21.15.6:38148) 2023-11-17T00:12:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd259c700 1 -- 172.21.15.6:0/786476589 learned_addr learned my addr 172.21.15.6:0/786476589 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:16.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 -- 172.21.15.6:0/786476589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 msgr2=0x562da78377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 -- 172.21.15.6:0/786476589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 msgr2=0x562da7837d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 -- 172.21.15.6:0/786476589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4009c80 con 0x562da78368b0 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd259c700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7dd359e700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0090a0 tx=0x7f7dcc004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0d0c327305ba53 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:16.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.577+0000 7f7db6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0a1af0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7dcc004ad0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da782c3f0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dcc099500 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 10 client.26258 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x562da75b0de0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562da782c3f0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7dcc0b92b0 con 0x562da78368b0 2023-11-17T00:12:16.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 10 client.26258 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 20 client.26258 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da789ff00 0x562da78a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 4 client.26258 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.578+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f7dcc0999c0 con 0x562da789ff00 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.579+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da789ff00 0x562da78a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.579+0000 7f7dd2d9d700 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da789ff00 0x562da78a22e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66a1c460727cd21 server_cookie=c79177c2a05882eb in_seq=0 out_seq=0 2023-11-17T00:12:16.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.579+0000 7f7db6ff5700 10 client.26258 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:16.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7db6ff5700 1 -- 172.21.15.6:0/786476589 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x7f7dcc0999c0 con 0x562da789ff00 2023-11-17T00:12:16.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7db6ff5700 10 client.26258 handle_command_reply: tid=0 2023-11-17T00:12:16.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 client.26258 shutdown 2023-11-17T00:12:16.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 msgr2=0x562da7833190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0090a0 tx=0x7f7dcc004cc0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 shutdown_connections 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562da78368b0 0x562da7833190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562da7831b60 0x562da78377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x562da789ff00 0x562da78a22e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 --2- 172.21.15.6:0/786476589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562da7832470 0x562da7837d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 >> 172.21.15.6:0/786476589 conn(0x562da7827360 msgr2=0x562da7834a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7db6ff5700 0 client.26258 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 shutdown_connections 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 1 -- 172.21.15.6:0/786476589 wait complete. 2023-11-17T00:12:16.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.581+0000 7f7decbfab80 20 client.26258 trim_cache size 0 max 16384 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (4107 inodes in the stack)", 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "92abc497-addd-4efc-ae92-66cf79369617": { 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "92abc497-addd-4efc-ae92-66cf79369617", 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:12:16.590 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 msgr2=0x7f7dbc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 0x7f7dbc0a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b580 tx=0x7f7dc8004880 comp rx=0 tx=0).stop 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 msgr2=0x7f7dd419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00ef30 tx=0x7f7dcc00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 shutdown_connections 2023-11-17T00:12:16.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7dbc0a12c0 0x7f7dbc0a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7dd4105560 0x7f7dd419cf70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dd4100c50 0x7f7dd41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 --2- 172.21.15.6:0/2352488488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7dd41015f0 0x7f7dd4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 >> 172.21.15.6:0/2352488488 conn(0x7f7dd4077500 msgr2=0x7f7dd40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 shutdown_connections 2023-11-17T00:12:16.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:16.582+0000 7f7dd91ff700 1 -- 172.21.15.6:0/2352488488 wait complete. 2023-11-17T00:12:16.603 INFO:tasks.cephfs.filesystem:scrub status for tag:92abc497-addd-4efc-ae92-66cf79369617 - {'path': '/', 'tag': '92abc497-addd-4efc-ae92-66cf79369617', 'options': 'recursive,force'} 2023-11-17T00:12:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:16 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:16 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:12:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:16 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:12:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:16 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:16 smithi087 ceph-mon[131182]: pgmap v104: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 2.66k op/s 2023-11-17T00:12:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:16 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:16 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:12:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:16 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:12:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:16 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:16 smithi186 ceph-mon[131487]: pgmap v104: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 2.66k op/s 2023-11-17T00:12:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:16 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:16 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:12:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:16 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:12:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:16 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:12:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:16 smithi006 ceph-mon[97840]: pgmap v104: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 2.66k op/s 2023-11-17T00:12:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 -- 172.21.15.6:0/2895973461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 msgr2=0x7f56d8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 --2- 172.21.15.6:0/2895973461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8104e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009990 tx=0x7f56c4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 -- 172.21.15.6:0/2895973461 shutdown_connections 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 --2- 172.21.15.6:0/2895973461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 0x7f56d8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 --2- 172.21.15.6:0/2895973461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 0x7f56d810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 --2- 172.21.15.6:0/2895973461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8104e20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 -- 172.21.15.6:0/2895973461 >> 172.21.15.6:0/2895973461 conn(0x7f56d80fbd10 msgr2=0x7f56d80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 -- 172.21.15.6:0/2895973461 shutdown_connections 2023-11-17T00:12:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.030+0000 7f56debff700 1 -- 172.21.15.6:0/2895973461 wait complete. 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.031+0000 7f56debff700 1 Processor -- start 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 -- start start 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 0x7f56d8072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 0x7f56d81a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8073b60 con 0x7f56d81004c0 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d81a4b70 con 0x7f56d8100e60 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56debff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d81a4cb0 con 0x7f56d8105500 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38166/0 (socket says 172.21.15.6:38166) 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56d7fff700 1 -- 172.21.15.6:0/3341398343 learned_addr learned my addr 172.21.15.6:0/3341398343 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56dc99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 0x7f56d8072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.032+0000 7f56dd19c700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 0x7f56d81a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 -- 172.21.15.6:0/3341398343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 msgr2=0x7f56d81a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 0x7f56d81a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 -- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 msgr2=0x7f56d8072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 0x7f56d8072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 -- 172.21.15.6:0/3341398343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40095f0 con 0x7f56d8100e60 2023-11-17T00:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d7fff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56c4004710 tx=0x7f56c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d5ffb700 1 -- 172.21.15.6:0/3341398343 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c401d070 con 0x7f56d8100e60 2023-11-17T00:12:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d5ffb700 1 -- 172.21.15.6:0/3341398343 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56c4021410 con 0x7f56d8100e60 2023-11-17T00:12:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d81102d0 con 0x7f56d8100e60 2023-11-17T00:12:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56d5ffb700 1 -- 172.21.15.6:0/3341398343 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4017650 con 0x7f56d8100e60 2023-11-17T00:12:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.033+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d8110140 con 0x7f56d8100e60 2023-11-17T00:12:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.037+0000 7f56d5ffb700 1 -- 172.21.15.6:0/3341398343 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f56c4021580 con 0x7f56d8100e60 2023-11-17T00:12:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.038+0000 7f56f246eb80 1 Processor -- start 2023-11-17T00:12:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.038+0000 7f56f246eb80 1 -- start start 2023-11-17T00:12:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 0x55f0770bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 0x55f0770b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f076fbf000 con 0x55f0770bf720 2023-11-17T00:12:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f077011840 con 0x55f076f156b0 2023-11-17T00:12:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56f246eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07700acd0 con 0x55f0770bfb90 2023-11-17T00:12:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56d5ffb700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 0x7f56c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56dd19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 0x55f0770bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56dd19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 0x55f0770bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38172/0 (socket says 172.21.15.6:38172) 2023-11-17T00:12:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56dd19c700 1 -- 172.21.15.6:0/1810741549 learned_addr learned my addr 172.21.15.6:0/1810741549 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 0x55f0770b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56d5ffb700 1 -- 172.21.15.6:0/3341398343 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56c40ce280 con 0x7f56d8100e60 2023-11-17T00:12:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.039+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dd19c700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 0x7f56c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 -- 172.21.15.6:0/1810741549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 msgr2=0x55f0770b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 -- 172.21.15.6:0/1810741549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 msgr2=0x55f0770bf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 0x55f0770bf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 -- 172.21.15.6:0/1810741549 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f076e8c5d0 con 0x55f0770bf720 2023-11-17T00:12:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 0x55f0770b9560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009cb0 tx=0x7f56cc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea9fe8ffa68ff62 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56dd19c700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 0x7f56c80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f56d00060b0 tx=0x7f56d0004b40 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56c2ffd700 1 -- 172.21.15.6:0/1810741549 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc011c60 con 0x55f0770bf720 2023-11-17T00:12:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56c2ffd700 1 -- 172.21.15.6:0/1810741549 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc0044f0 con 0x55f0770bf720 2023-11-17T00:12:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.040+0000 7f56c2ffd700 1 -- 172.21.15.6:0/1810741549 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc005280 con 0x55f0770bf720 2023-11-17T00:12:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 -- 172.21.15.6:0/1810741549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 msgr2=0x55f0770b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 0x55f0770b9560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009cb0 tx=0x7f56cc011440 comp rx=0 tx=0).stop 2023-11-17T00:12:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 -- 172.21.15.6:0/1810741549 shutdown_connections 2023-11-17T00:12:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f076f156b0 0x55f0770bf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1810741549 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bf720 0x55f0770b9560 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 -- 172.21.15.6:0/1810741549 >> 172.21.15.6:0/1810741549 conn(0x55f0770aef50 msgr2=0x55f0770b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 -- 172.21.15.6:0/1810741549 shutdown_connections 2023-11-17T00:12:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.041+0000 7f56f246eb80 1 -- 172.21.15.6:0/1810741549 wait complete. 2023-11-17T00:12:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.042+0000 7f56f246eb80 1 -- start start 2023-11-17T00:12:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.042+0000 7f56f246eb80 1 Processor -- start 2023-11-17T00:12:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f076fac880 con 0x55f076f156b0 2023-11-17T00:12:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f076fac520 con 0x55f0770bf720 2023-11-17T00:12:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f076e8e8c0 con 0x55f0770bfb90 2023-11-17T00:12:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56f246eb80 10 client.0 _ll_register_callbacks cb 0x55f076fca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56dc99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38186/0 (socket says 172.21.15.6:38186) 2023-11-17T00:12:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 -- 172.21.15.6:0/450968804 learned_addr learned my addr 172.21.15.6:0/450968804 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56dd19c700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 -- 172.21.15.6:0/450968804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 msgr2=0x55f0770c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 -- 172.21.15.6:0/450968804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 msgr2=0x55f0770c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56d7fff700 1 -- 172.21.15.6:0/450968804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc0098e0 con 0x55f0770bf720 2023-11-17T00:12:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.043+0000 7f56dc99b700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56d7fff700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f56c409ae40 tx=0x7f56c4021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa9c73ae809aa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c409b140 con 0x55f0770bf720 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56c409f170 con 0x55f0770bf720 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0770ac710 con 0x55f0770bf720 2023-11-17T00:12:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c40a3500 con 0x55f0770bf720 2023-11-17T00:12:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56f246eb80 10 client.26270 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f076fd1130 con 0x55f0770bf720 2023-11-17T00:12:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.044+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56c40b7080 con 0x55f0770bf720 2023-11-17T00:12:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f56c4096f30 con 0x55f0770bf720 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 10 client.26270 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 20 client.26270 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f0770d2be0 0x55f0770d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 4 client.26270 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:12:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f076fcd040 con 0x55f0770d2be0 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56dc99b700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f0770d2be0 0x55f0770d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56dc99b700 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f0770d2be0 0x55f0770d4ff0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ca77a89fc31a4a server_cookie=2d308b67e75f35b7 in_seq=0 out_seq=0 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.045+0000 7f56b7fff700 10 client.26270 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.047+0000 7f56b7fff700 1 -- 172.21.15.6:0/450968804 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f076fcd040 con 0x55f0770d2be0 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.047+0000 7f56b7fff700 10 client.26270 handle_command_reply: tid=0 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.047+0000 7f56f246eb80 1 client.26270 shutdown 2023-11-17T00:12:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.047+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 msgr2=0x55f0770c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.047+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f56c409ae40 tx=0x7f56c4021b60 comp rx=0 tx=0).stop 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 shutdown_connections 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f0770d2be0 0x55f0770d4ff0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 --2- 172.21.15.6:0/450968804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c2d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56b7fff700 0 client.26270 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 >> 172.21.15.6:0/450968804 conn(0x55f0770aef50 msgr2=0x55f0770b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 shutdown_connections 2023-11-17T00:12:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 1 -- 172.21.15.6:0/450968804 wait complete. 2023-11-17T00:12:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.048+0000 7f56f246eb80 20 client.26270 trim_cache size 0 max 16384 2023-11-17T00:12:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.064+0000 7f56f246eb80 1 Processor -- start 2023-11-17T00:12:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.064+0000 7f56f246eb80 1 -- start start 2023-11-17T00:12:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.064+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.064+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.064+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56f246eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0770bedf0 con 0x55f076f156b0 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56f246eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0770bda10 con 0x55f0770bf720 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56f246eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0770b9e20 con 0x55f0770bfb90 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56dc99b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38188/0 (socket says 172.21.15.6:38188) 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 -- 172.21.15.6:0/1719308768 learned_addr learned my addr 172.21.15.6:0/1719308768 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56dd19c700 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 -- 172.21.15.6:0/1719308768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 msgr2=0x55f0770c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 -- 172.21.15.6:0/1719308768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 msgr2=0x55f0770ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 -- 172.21.15.6:0/1719308768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0770b06f0 con 0x55f0770bf720 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.065+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f56c4097bb0 tx=0x7f56c4005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d4191456283cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56c37fe700 1 -- 172.21.15.6:0/1719308768 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c409f170 con 0x55f0770bf720 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56c37fe700 1 -- 172.21.15.6:0/1719308768 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0770ac710 con 0x55f0770bf720 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56c37fe700 1 -- 172.21.15.6:0/1719308768 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a3070 con 0x55f0770bf720 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 -- 172.21.15.6:0/1719308768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 msgr2=0x55f0770b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f56c4097bb0 tx=0x7f56c4005a90 comp rx=0 tx=0).stop 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 -- 172.21.15.6:0/1719308768 shutdown_connections 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f076f156b0 0x55f0770ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f0770bfb90 0x55f0770c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1719308768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770b3f60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 -- 172.21.15.6:0/1719308768 >> 172.21.15.6:0/1719308768 conn(0x55f0770aef50 msgr2=0x55f0770bcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 -- 172.21.15.6:0/1719308768 shutdown_connections 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.066+0000 7f56f246eb80 1 -- 172.21.15.6:0/1719308768 wait complete. 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.067+0000 7f56f246eb80 1 -- start start 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.067+0000 7f56f246eb80 1 Processor -- start 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.067+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f076f156b0 0x55f0770c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.067+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56f246eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bfb90 0x55f0770c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56f246eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f077125a50 con 0x55f0770bfb90 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56f246eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f077125b90 con 0x55f0770bf720 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56f246eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f077125d50 con 0x55f076f156b0 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56f246eb80 10 client.0 _ll_register_callbacks cb 0x7f56c4097510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:38202/0 (socket says 172.21.15.6:38202) 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 -- 172.21.15.6:0/1117433888 learned_addr learned my addr 172.21.15.6:0/1117433888 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f076f156b0 0x55f0770c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56dd19c700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bfb90 0x55f0770c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 -- 172.21.15.6:0/1117433888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f076f156b0 msgr2=0x55f0770c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f076f156b0 0x55f0770c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 -- 172.21.15.6:0/1117433888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bfb90 msgr2=0x55f0770c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bfb90 0x55f0770c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 -- 172.21.15.6:0/1117433888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc01d040 con 0x55f0770bf720 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56d7fff700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f56c4099490 tx=0x7f56c4005860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a730499a0cfe6a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.068+0000 7f56b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40a3270 con 0x55f0770bf720 2023-11-17T00:12:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56c409b140 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56cc0044f0 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c40a3500 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 10 client.26276 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f076fd1130 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc0044f0 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f56c40b3a90 con 0x55f0770bf720 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 10 client.26276 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 20 client.26276 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f077127010 0x55f077129420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 4 client.26276 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:12:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.069+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f076fd9600 con 0x55f077127010 2023-11-17T00:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.070+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f077127010 0x55f077129420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.070+0000 7f56dc99b700 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f077127010 0x55f077129420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba66842057643de server_cookie=d604f8f668d986c6 in_seq=0 out_seq=0 2023-11-17T00:12:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:18.070+0000 7f56b7fff700 10 client.26276 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:18 smithi087 ceph-mon[131182]: pgmap v105: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:18 smithi186 ceph-mon[131487]: pgmap v105: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:18 smithi006 ceph-mon[97840]: pgmap v105: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:20 smithi087 ceph-mon[131182]: pgmap v106: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:20 smithi186 ceph-mon[131487]: pgmap v106: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:21.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:20 smithi006 ceph-mon[97840]: pgmap v106: 129 pgs: 129 active+clean; 53 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-11-17T00:12:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:22 smithi087 ceph-mon[131182]: pgmap v107: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 224 op/s 2023-11-17T00:12:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:22 smithi186 ceph-mon[131487]: pgmap v107: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 224 op/s 2023-11-17T00:12:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:22 smithi006 ceph-mon[97840]: pgmap v107: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 224 op/s 2023-11-17T00:12:25.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:24 smithi087 ceph-mon[131182]: pgmap v108: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 40 op/s 2023-11-17T00:12:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:24 smithi186 ceph-mon[131487]: pgmap v108: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 40 op/s 2023-11-17T00:12:25.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:24 smithi006 ceph-mon[97840]: pgmap v108: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 40 op/s 2023-11-17T00:12:27.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:26 smithi087 ceph-mon[131182]: pgmap v109: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 362 op/s 2023-11-17T00:12:27.137 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:26 smithi186 ceph-mon[131487]: pgmap v109: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 362 op/s 2023-11-17T00:12:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:26 smithi006 ceph-mon[97840]: pgmap v109: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.2 MiB/s wr, 362 op/s 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56b7fff700 1 -- 172.21.15.6:0/1117433888 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f076fd9600 con 0x55f077127010 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56b7fff700 10 client.26276 handle_command_reply: tid=0 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 client.26276 shutdown 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 msgr2=0x55f0770c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f56c4099490 tx=0x7f56c4005860 comp rx=0 tx=0).stop 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 shutdown_connections 2023-11-17T00:12:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55f077127010 0x55f077129420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f0770bfb90 0x55f0770c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f076f156b0 0x55f0770c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 --2- 172.21.15.6:0/1117433888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f0770bf720 0x55f0770c6b70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56b7fff700 0 client.26276 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 >> 172.21.15.6:0/1117433888 conn(0x55f0770aef50 msgr2=0x55f0770c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 shutdown_connections 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.817+0000 7f56f246eb80 1 -- 172.21.15.6:0/1117433888 wait complete. 2023-11-17T00:12:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.818+0000 7f56f246eb80 20 client.26276 trim_cache size 0 max 16384 2023-11-17T00:12:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:12:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:12:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:12:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:12:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.818+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 msgr2=0x7f56c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.818+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 0x7f56c80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f56d00060b0 tx=0x7f56d0004b40 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.818+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 msgr2=0x7f56d8073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.818+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56c4004710 tx=0x7f56c4009580 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 shutdown_connections 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f56c80a12c0 0x7f56c80a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56d81004c0 0x7f56d8072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56d8105500 0x7f56d81a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 --2- 172.21.15.6:0/3341398343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56d8100e60 0x7f56d8073330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 >> 172.21.15.6:0/3341398343 conn(0x7f56d80fbd10 msgr2=0x7f56d80fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 shutdown_connections 2023-11-17T00:12:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:27.819+0000 7f56debff700 1 -- 172.21.15.6:0/3341398343 wait complete. 2023-11-17T00:12:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:28 smithi087 ceph-mon[131182]: pgmap v110: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:28 smithi186 ceph-mon[131487]: pgmap v110: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:28 smithi006 ceph-mon[97840]: pgmap v110: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:31.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:30 smithi087 ceph-mon[131182]: pgmap v111: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:30 smithi186 ceph-mon[131487]: pgmap v111: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:31.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:30 smithi006 ceph-mon[97840]: pgmap v111: 129 pgs: 129 active+clean; 89 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2023-11-17T00:12:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:32 smithi087 ceph-mon[131182]: pgmap v112: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2023-11-17T00:12:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:32 smithi186 ceph-mon[131487]: pgmap v112: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2023-11-17T00:12:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:32 smithi006 ceph-mon[97840]: pgmap v112: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2023-11-17T00:12:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:34 smithi087 ceph-mon[131182]: pgmap v113: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 339 op/s 2023-11-17T00:12:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:34 smithi186 ceph-mon[131487]: pgmap v113: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 339 op/s 2023-11-17T00:12:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:34 smithi006 ceph-mon[97840]: pgmap v113: 129 pgs: 129 active+clean; 39 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 339 op/s 2023-11-17T00:12:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:36 smithi087 ceph-mon[131182]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-11-17T00:12:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:36 smithi186 ceph-mon[131487]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-11-17T00:12:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:36 smithi006 ceph-mon[97840]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-11-17T00:12:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:38 smithi087 ceph-mon[131182]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:39.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:38 smithi186 ceph-mon[131487]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:39.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:38 smithi006 ceph-mon[97840]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 -- 172.21.15.6:0/3729017708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101400 msgr2=0x7f4b681017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/3729017708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101400 0x7f4b681017d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009990 tx=0x7f4b64009ca0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 -- 172.21.15.6:0/3729017708 shutdown_connections 2023-11-17T00:12:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/3729017708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b6810a740 0x7f4b6810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/3729017708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101400 0x7f4b681017d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/3729017708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b68101d10 0x7f4b6810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.015+0000 7f4b6fdff700 1 -- 172.21.15.6:0/3729017708 >> 172.21.15.6:0/3729017708 conn(0x7f4b68076150 msgr2=0x7f4b68076550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.016+0000 7f4b6fdff700 1 -- 172.21.15.6:0/3729017708 shutdown_connections 2023-11-17T00:12:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.016+0000 7f4b6fdff700 1 -- 172.21.15.6:0/3729017708 wait complete. 2023-11-17T00:12:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.017+0000 7f4b6fdff700 1 Processor -- start 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.017+0000 7f4b6fdff700 1 -- start start 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 0x7f4b68198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 0x7f4b6819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6819d510 con 0x7f4b68101400 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6819d680 con 0x7f4b68101d10 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6819d840 con 0x7f4b6810a740 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43372/0 (socket says 172.21.15.6:43372) 2023-11-17T00:12:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6db9b700 1 -- 172.21.15.6:0/1507325326 learned_addr learned my addr 172.21.15.6:0/1507325326 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 0x7f4b68198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 0x7f4b6819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.018+0000 7f4b6db9b700 1 -- 172.21.15.6:0/1507325326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 msgr2=0x7f4b6819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 0x7f4b6819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6db9b700 1 -- 172.21.15.6:0/1507325326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 msgr2=0x7f4b68198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 0x7f4b68198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6db9b700 1 -- 172.21.15.6:0/1507325326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640095f0 con 0x7f4b68101400 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 0x7f4b68198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 0x7f4b6819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005910 tx=0x7f4b64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b5effd700 1 -- 172.21.15.6:0/1507325326 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6401d070 con 0x7f4b68101400 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b68110240 con 0x7f4b68101400 2023-11-17T00:12:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b681100b0 con 0x7f4b68101400 2023-11-17T00:12:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b5effd700 1 -- 172.21.15.6:0/1507325326 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64022470 con 0x7f4b68101400 2023-11-17T00:12:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.019+0000 7f4b5effd700 1 -- 172.21.15.6:0/1507325326 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64017650 con 0x7f4b68101400 2023-11-17T00:12:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.023+0000 7f4b5effd700 1 -- 172.21.15.6:0/1507325326 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4b6400fad0 con 0x7f4b68101400 2023-11-17T00:12:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.025+0000 7f4b837efb80 1 Processor -- start 2023-11-17T00:12:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.025+0000 7f4b837efb80 1 -- start start 2023-11-17T00:12:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e213456b0 0x559e214ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e214e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b5effd700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 0x7f4b540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e213ef000 con 0x559e213456b0 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e21441840 con 0x559e214ef720 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b837efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2143acd0 con 0x559e214efb90 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b5effd700 1 -- 172.21.15.6:0/1507325326 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4b640ce4b0 con 0x7f4b68101400 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52780/0 (socket says 172.21.15.6:52780) 2023-11-17T00:12:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.026+0000 7f4b6db9b700 1 -- 172.21.15.6:0/741373823 learned_addr learned my addr 172.21.15.6:0/741373823 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e214e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 0x7f4b540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 -- 172.21.15.6:0/741373823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 msgr2=0x559e214e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e214e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 -- 172.21.15.6:0/741373823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e213456b0 msgr2=0x559e214ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e213456b0 0x559e214ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 -- 172.21.15.6:0/741373823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e212bc5d0 con 0x559e214ef720 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005f50 tx=0x7f4b64005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f9529d635ad10c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.027+0000 7f4b53fff700 1 -- 172.21.15.6:0/741373823 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640b3d90 con 0x559e214ef720 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b53fff700 1 -- 172.21.15.6:0/741373823 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b640b3bb0 con 0x559e214ef720 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b53fff700 1 -- 172.21.15.6:0/741373823 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400fd80 con 0x559e214ef720 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 -- 172.21.15.6:0/741373823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 msgr2=0x559e214e9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005f50 tx=0x7f4b64005c70 comp rx=0 tx=0).stop 2023-11-17T00:12:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 0x7f4b540a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b600060b0 tx=0x7f4b60009130 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 -- 172.21.15.6:0/741373823 shutdown_connections 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e213456b0 0x559e214ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e214e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 --2- 172.21.15.6:0/741373823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e214ef720 0x559e214e9560 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.028+0000 7f4b837efb80 1 -- 172.21.15.6:0/741373823 >> 172.21.15.6:0/741373823 conn(0x559e214def50 msgr2=0x559e214e4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.029+0000 7f4b837efb80 1 -- 172.21.15.6:0/741373823 shutdown_connections 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.029+0000 7f4b837efb80 1 -- 172.21.15.6:0/741373823 wait complete. 2023-11-17T00:12:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.029+0000 7f4b837efb80 1 -- start start 2023-11-17T00:12:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.029+0000 7f4b837efb80 1 Processor -- start 2023-11-17T00:12:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e213dc880 con 0x559e214efb90 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e213dc520 con 0x559e213456b0 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e212be8c0 con 0x559e214ef720 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b837efb80 10 client.0 _ll_register_callbacks cb 0x559e213fa720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52794/0 (socket says 172.21.15.6:52794) 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b6db9b700 1 -- 172.21.15.6:0/3375864372 learned_addr learned my addr 172.21.15.6:0/3375864372 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.030+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 -- 172.21.15.6:0/3375864372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 msgr2=0x559e214f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 -- 172.21.15.6:0/3375864372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 msgr2=0x559e214f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 -- 172.21.15.6:0/3375864372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b681100b0 con 0x559e213456b0 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b6409b1f0 tx=0x7f4b6409f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b179ef0cb5529f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400fd80 con 0x559e213456b0 2023-11-17T00:12:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e214dc710 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.031+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4b640b63e0 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6409f020 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 10 client.26288 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559e21401130 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e214dc710 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b6409b530 con 0x559e213456b0 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 10 client.26288 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 20 client.26288 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21502710 0x559e21504b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 4 client.26288 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.032+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e213fd040 con 0x559e21502710 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.033+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21502710 0x559e21504b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.033+0000 7f4b6d39a700 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21502710 0x559e21504b20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9f159ee21063dd server_cookie=6eff5e07c2f0bec4 in_seq=0 out_seq=0 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.034+0000 7f4b517fa700 10 client.26288 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.034+0000 7f4b517fa700 1 -- 172.21.15.6:0/3375864372 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559e213fd040 con 0x559e21502710 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.034+0000 7f4b517fa700 10 client.26288 handle_command_reply: tid=0 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 client.26288 shutdown 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 msgr2=0x559e214f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b6409b1f0 tx=0x7f4b6409f380 comp rx=0 tx=0).stop 2023-11-17T00:12:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 shutdown_connections 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21502710 0x559e21504b20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b517fa700 0 client.26288 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f86d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 --2- 172.21.15.6:0/3375864372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 >> 172.21.15.6:0/3375864372 conn(0x559e214def50 msgr2=0x559e214e37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 shutdown_connections 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 1 -- 172.21.15.6:0/3375864372 wait complete. 2023-11-17T00:12:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.035+0000 7f4b837efb80 20 client.26288 trim_cache size 0 max 16384 2023-11-17T00:12:41.122 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:40 smithi186 ceph-mon[131487]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:41.122 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:40 smithi087 ceph-mon[131182]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.054+0000 7f4b837efb80 1 Processor -- start 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 -- start start 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e214e5c50 con 0x559e214efb90 2023-11-17T00:12:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e21505180 con 0x559e213456b0 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b837efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e214f0fa0 con 0x559e214ef720 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b6d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b6db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52806/0 (socket says 172.21.15.6:52806) 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b6d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59836/0 (socket says 172.21.15.6:59836) 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.055+0000 7f4b6db9b700 1 -- 172.21.15.6:0/2582469896 learned_addr learned my addr 172.21.15.6:0/2582469896 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 -- 172.21.15.6:0/2582469896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 msgr2=0x559e214e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214e3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 -- 172.21.15.6:0/2582469896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 msgr2=0x559e214ebc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214ebc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 -- 172.21.15.6:0/2582469896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e21400e80 con 0x559e213456b0 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b6409af10 tx=0x7f4b64005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c2b3aa783ae3f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b5cbff700 1 -- 172.21.15.6:0/2582469896 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400fd80 con 0x559e213456b0 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b5cbff700 1 -- 172.21.15.6:0/2582469896 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6409a8f0 con 0x559e213456b0 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.056+0000 7f4b5cbff700 1 -- 172.21.15.6:0/2582469896 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640b63e0 con 0x559e213456b0 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 -- 172.21.15.6:0/2582469896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 msgr2=0x559e214f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4b6409af10 tx=0x7f4b64005c70 comp rx=0 tx=0).stop 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 -- 172.21.15.6:0/2582469896 shutdown_connections 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214efb90 0x559e214ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f29b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 --2- 172.21.15.6:0/2582469896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214ef720 0x559e214e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 -- 172.21.15.6:0/2582469896 >> 172.21.15.6:0/2582469896 conn(0x559e214def50 msgr2=0x559e214f22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 -- 172.21.15.6:0/2582469896 shutdown_connections 2023-11-17T00:12:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.057+0000 7f4b837efb80 1 -- 172.21.15.6:0/2582469896 wait complete. 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.058+0000 7f4b837efb80 1 -- start start 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.058+0000 7f4b837efb80 1 Processor -- start 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 0x559e214f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e2154d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2154dcf0 con 0x559e214ef720 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2154de60 con 0x559e213456b0 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2154e020 con 0x559e214efb90 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b837efb80 10 client.0 _ll_register_callbacks cb 0x7f4b58009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b6d39a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 0x559e214f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b6e39c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e2154d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b6d39a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 0x559e214f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43390/0 (socket says 172.21.15.6:43390) 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.059+0000 7f4b6d39a700 1 -- 172.21.15.6:0/1104655735 learned_addr learned my addr 172.21.15.6:0/1104655735 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 -- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 msgr2=0x559e214f7cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 -- 172.21.15.6:0/1104655735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 msgr2=0x559e214f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 0x559e214f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 -- 172.21.15.6:0/1104655735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e21401130 con 0x559e214efb90 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b6e39c700 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e2154d780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b600092f0 tx=0x7f4b60006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ecd1045d6a0cd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6001db60 con 0x559e214efb90 2023-11-17T00:12:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4b6001dcc0 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.060+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b640b3bb0 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6001dea0 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 10 client.26251 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559e21251c00 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b640b3bb0 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b60027530 con 0x559e214efb90 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 10 client.26251 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 20 client.26251 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21556af0 0x559e21558f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 4 client.26251 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:12:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.061+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e21409600 con 0x559e21556af0 2023-11-17T00:12:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.062+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21556af0 0x559e21558f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.062+0000 7f4b517fa700 10 client.26251 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:41.062+0000 7f4b6db9b700 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21556af0 0x559e21558f00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf686dca6488fd8 server_cookie=b6aa3bf869e847e4 in_seq=0 out_seq=0 2023-11-17T00:12:41.239 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:40 smithi006 ceph-mon[97840]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 143 op/s 2023-11-17T00:12:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:42 smithi087 ceph-mon[131182]: pgmap v117: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-11-17T00:12:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:42 smithi186 ceph-mon[131487]: pgmap v117: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-11-17T00:12:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:42 smithi006 ceph-mon[97840]: pgmap v117: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-11-17T00:12:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.425+0000 7f4b517fa700 1 -- 172.21.15.6:0/1104655735 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e21409600 con 0x559e21556af0 2023-11-17T00:12:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.425+0000 7f4b517fa700 10 client.26251 handle_command_reply: tid=0 2023-11-17T00:12:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.425+0000 7f4b837efb80 1 client.26251 shutdown 2023-11-17T00:12:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 msgr2=0x559e2154d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e2154d780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b600092f0 tx=0x7f4b60006040 comp rx=0 tx=0).stop 2023-11-17T00:12:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 shutdown_connections 2023-11-17T00:12:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559e214efb90 0x559e2154d780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559e213456b0 0x559e214f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559e21556af0 0x559e21558f00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 --2- 172.21.15.6:0/1104655735 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559e214ef720 0x559e214f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 >> 172.21.15.6:0/1104655735 conn(0x559e214def50 msgr2=0x559e214f11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.426+0000 7f4b517fa700 0 client.26251 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.427+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 shutdown_connections 2023-11-17T00:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.427+0000 7f4b837efb80 1 -- 172.21.15.6:0/1104655735 wait complete. 2023-11-17T00:12:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.428+0000 7f4b837efb80 20 client.26251 trim_cache size 0 max 16384 2023-11-17T00:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:12:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:12:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:12:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.429+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 msgr2=0x7f4b540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.429+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 0x7f4b540a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b600060b0 tx=0x7f4b60009130 comp rx=0 tx=0).stop 2023-11-17T00:12:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.429+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 msgr2=0x7f4b68198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.429+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005910 tx=0x7f4b64004730 comp rx=0 tx=0).stop 2023-11-17T00:12:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 shutdown_connections 2023-11-17T00:12:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f4b540a1340 0x7f4b540a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b68101400 0x7f4b68198530 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b6810a740 0x7f4b6819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 --2- 172.21.15.6:0/1507325326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b68101d10 0x7f4b68198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 >> 172.21.15.6:0/1507325326 conn(0x7f4b68076150 msgr2=0x7f4b680ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 shutdown_connections 2023-11-17T00:12:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:44.430+0000 7f4b6fdff700 1 -- 172.21.15.6:0/1507325326 wait complete. 2023-11-17T00:12:45.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:44 smithi087 ceph-mon[131182]: pgmap v118: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-11-17T00:12:45.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:44 smithi186 ceph-mon[131487]: pgmap v118: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-11-17T00:12:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:44 smithi006 ceph-mon[97840]: pgmap v118: 129 pgs: 129 active+clean; 68 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2023-11-17T00:12:46.604 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:12:47.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/4282397909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 msgr2=0x7f8c7410f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/4282397909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c7410f230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b0a0 tx=0x7f8c6c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/4282397909 shutdown_connections 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/4282397909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c7410f230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/4282397909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c740748a0 0x7f8c74072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/4282397909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c74073260 0x7f8c740736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/4282397909 >> 172.21.15.6:0/4282397909 conn(0x7f8c74077530 msgr2=0x7f8c74077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/4282397909 shutdown_connections 2023-11-17T00:12:47.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.064+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/4282397909 wait complete. 2023-11-17T00:12:47.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.066+0000 7f8c7b5ff700 1 Processor -- start 2023-11-17T00:12:47.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.066+0000 7f8c7b5ff700 1 -- start start 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c74073260 0x7f8c7419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 0x7f8c7419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74198da0 con 0x7f8c740748a0 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74198f10 con 0x7f8c7410ce50 2023-11-17T00:12:47.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7b5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741990d0 con 0x7f8c74073260 2023-11-17T00:12:47.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c74073260 0x7f8c7419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52836/0 (socket says 172.21.15.6:52836) 2023-11-17T00:12:47.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.067+0000 7f8c79b9c700 1 -- 172.21.15.6:0/2047205283 learned_addr learned my addr 172.21.15.6:0/2047205283 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:47.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.068+0000 7f8c79b9c700 1 -- 172.21.15.6:0/2047205283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c74073260 msgr2=0x7f8c7419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.068+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c74073260 0x7f8c7419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.068+0000 7f8c79b9c700 1 -- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 msgr2=0x7f8c7419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.068+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 0x7f8c7419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.069+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 0x7f8c7419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.069+0000 7f8c79b9c700 1 -- 172.21.15.6:0/2047205283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c009d00 con 0x7f8c7410ce50 2023-11-17T00:12:47.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.069+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 0x7f8c7419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.070+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009fd0 tx=0x7f8c6c00ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.070+0000 7f8c6a7fc700 1 -- 172.21.15.6:0/2047205283 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00e040 con 0x7f8c7410ce50 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.070+0000 7f8c6a7fc700 1 -- 172.21.15.6:0/2047205283 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c004de0 con 0x7f8c7410ce50 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.070+0000 7f8c6a7fc700 1 -- 172.21.15.6:0/2047205283 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c01b6b0 con 0x7f8c7410ce50 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.071+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c741127b0 con 0x7f8c7410ce50 2023-11-17T00:12:47.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.071+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c74112620 con 0x7f8c7410ce50 2023-11-17T00:12:47.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.077+0000 7f8c8eee7b80 1 Processor -- start 2023-11-17T00:12:47.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.077+0000 7f8c6a7fc700 1 -- 172.21.15.6:0/2047205283 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8c6c003a00 con 0x7f8c7410ce50 2023-11-17T00:12:47.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.078+0000 7f8c8eee7b80 1 -- start start 2023-11-17T00:12:47.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d1e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d0dc060 con 0x55e93d1dfc80 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d12e8a0 con 0x55e93d1df370 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d127d30 con 0x55e93d1e40c0 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c78b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c78b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52842/0 (socket says 172.21.15.6:52842) 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43418/0 (socket says 172.21.15.6:43418) 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.079+0000 7f8c78b9a700 1 -- 172.21.15.6:0/1769916697 learned_addr learned my addr 172.21.15.6:0/1769916697 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d1e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c7939b700 1 -- 172.21.15.6:0/1769916697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 msgr2=0x55e93d1e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c7939b700 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d1e4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c7939b700 1 -- 172.21.15.6:0/1769916697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 msgr2=0x55e93d1df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c7939b700 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.080+0000 7f8c7939b700 1 -- 172.21.15.6:0/1769916697 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93cfa9630 con 0x55e93d1dfc80 2023-11-17T00:12:47.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.081+0000 7f8c6a7fc700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 0x7f8c600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.081+0000 7f8c7939b700 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bc80 tx=0x7f8c7000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1c0b3fb9ddc175 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.081+0000 7f8c6a7fc700 1 -- 172.21.15.6:0/2047205283 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c6c0ce890 con 0x7f8c7410ce50 2023-11-17T00:12:47.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.082+0000 7f8c5f7fe700 1 -- 172.21.15.6:0/1769916697 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70010ca0 con 0x55e93d1dfc80 2023-11-17T00:12:47.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.082+0000 7f8c5f7fe700 1 -- 172.21.15.6:0/1769916697 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70015070 con 0x55e93d1dfc80 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.082+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 0x7f8c600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.082+0000 7f8c5f7fe700 1 -- 172.21.15.6:0/1769916697 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70004e70 con 0x55e93d1dfc80 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.083+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/1769916697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 msgr2=0x55e93d1e3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.083+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bc80 tx=0x7f8c7000bf90 comp rx=0 tx=0).stop 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.083+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 0x7f8c600a3870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400afd0 tx=0x7f8c6400a1f0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.084+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/1769916697 shutdown_connections 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.085+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d1e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.085+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.085+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/1769916697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e3b80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.085+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/1769916697 >> 172.21.15.6:0/1769916697 conn(0x55e93d1cc360 msgr2=0x55e93d1d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.086+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/1769916697 shutdown_connections 2023-11-17T00:12:47.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.086+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/1769916697 wait complete. 2023-11-17T00:12:47.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.086+0000 7f8c8eee7b80 1 -- start start 2023-11-17T00:12:47.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.087+0000 7f8c8eee7b80 1 Processor -- start 2023-11-17T00:12:47.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.088+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 0x55e93d1eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.088+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.088+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52848/0 (socket says 172.21.15.6:52848) 2023-11-17T00:12:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c78b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3829812591 learned_addr learned my addr 172.21.15.6:0/3829812591 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c7939b700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 0x55e93d1eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d0c98e0 con 0x55e93d1dfc80 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d0c9580 con 0x55e93d1e40c0 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.089+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cfab920 con 0x55e93d1df370 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3829812591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 msgr2=0x55e93d1eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c8eee7b80 10 client.0 _ll_register_callbacks cb 0x55e93d0e7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 0x55e93d1eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3829812591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 msgr2=0x55e93d1e8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7000b8e0 con 0x55e93d1e40c0 2023-11-17T00:12:47.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c7939b700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 0x55e93d1eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e93d1c9b20 con 0x55e93d1e40c0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c09b5d0 tx=0x7f8c6c09b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668615a43fd6ded0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.090+0000 7f8c5cff9700 10 client.26303 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c5cff9700 1 -- 172.21.15.6:0/3829812591 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0cc0b0 con 0x55e93d1e40c0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c5cff9700 1 -- 172.21.15.6:0/3829812591 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c6c084e40 con 0x55e93d1e40c0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c5cff9700 1 -- 172.21.15.6:0/3829812591 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c004880 con 0x55e93d1e40c0 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 10 client.26303 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:47.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({fsmap=22}) v3 -- 0x55e93d0ee190 con 0x55e93d1e40c0 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c5cff9700 1 -- 172.21.15.6:0/3829812591 <== mon.2 v2:172.21.15.87:3300/0 4 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c6c0b0020 con 0x55e93d1e40c0 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 10 client.26303 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 20 client.26303 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d1f8c30 0x55e93d1fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 4 client.26303 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.091+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e93d0ea0a0 con 0x55e93d1f8c30 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.092+0000 7f8c7939b700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d1f8c30 0x55e93d1fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.092+0000 7f8c5cff9700 10 client.26303 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.092+0000 7f8c7939b700 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d1f8c30 0x55e93d1fb040 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4377fc6de713f1b server_cookie=521fca108198351b in_seq=0 out_seq=0 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.093+0000 7f8c5cff9700 1 -- 172.21.15.6:0/3829812591 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e93d0ea0a0 con 0x55e93d1f8c30 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c5cff9700 10 client.26303 handle_command_reply: tid=0 2023-11-17T00:12:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 client.26303 shutdown 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 msgr2=0x55e93d1e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c09b5d0 tx=0x7f8c6c09b4c0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 shutdown_connections 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1e9040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1df370 0x55e93d1eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d1f8c30 0x55e93d1fb040 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c5cff9700 0 client.26303 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3829812591 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.094+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 >> 172.21.15.6:0/3829812591 conn(0x55e93d1cc360 msgr2=0x55e93d1e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.095+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 shutdown_connections 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.095+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3829812591 wait complete. 2023-11-17T00:12:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.095+0000 7f8c8eee7b80 20 client.26303 trim_cache size 0 max 16384 2023-11-17T00:12:47.104 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:46 smithi087 ceph-mon[131182]: pgmap v119: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 1.45k op/s 2023-11-17T00:12:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:46 smithi186 ceph-mon[131487]: pgmap v119: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 1.45k op/s 2023-11-17T00:12:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.120+0000 7f8c8eee7b80 1 Processor -- start 2023-11-17T00:12:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.120+0000 7f8c8eee7b80 1 -- start start 2023-11-17T00:12:47.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.120+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1df370 0x55e93d1d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.120+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1dfc80 0x55e93d1f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.120+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d1d27e0 con 0x55e93d1df370 2023-11-17T00:12:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d1d29a0 con 0x55e93d1e40c0 2023-11-17T00:12:47.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d1e03d0 con 0x55e93d1dfc80 2023-11-17T00:12:47.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1df370 0x55e93d1d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52854/0 (socket says 172.21.15.6:52854) 2023-11-17T00:12:47.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3081789746 learned_addr learned my addr 172.21.15.6:0/3081789746 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1dfc80 0x55e93d1f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3081789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1dfc80 msgr2=0x55e93d1f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1dfc80 0x55e93d1f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3081789746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1df370 msgr2=0x55e93d1d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1df370 0x55e93d1d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.121+0000 7f8c79b9c700 1 -- 172.21.15.6:0/3081789746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93d1d0820 con 0x55e93d1e40c0 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.122+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c09b340 tx=0x7f8c6c0bb050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967edba5eb13d5e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.122+0000 7f8c5ffff700 1 -- 172.21.15.6:0/3081789746 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004880 con 0x55e93d1e40c0 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.122+0000 7f8c5ffff700 1 -- 172.21.15.6:0/3081789746 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e93d1c9b20 con 0x55e93d1e40c0 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.122+0000 7f8c5ffff700 1 -- 172.21.15.6:0/3081789746 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0860b0 con 0x55e93d1e40c0 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3081789746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 msgr2=0x55e93d1de170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c09b340 tx=0x7f8c6c0bb050 comp rx=0 tx=0).stop 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3081789746 shutdown_connections 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1df370 0x55e93d1d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1e40c0 0x55e93d1de170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/3081789746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1dfc80 0x55e93d1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.123+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3081789746 >> 172.21.15.6:0/3081789746 conn(0x55e93d1cc360 msgr2=0x55e93d1f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.124+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3081789746 shutdown_connections 2023-11-17T00:12:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.124+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/3081789746 wait complete. 2023-11-17T00:12:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.124+0000 7f8c8eee7b80 1 -- start start 2023-11-17T00:12:47.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 Processor -- start 2023-11-17T00:12:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d245ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d246030 con 0x55e93d1dfc80 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d2461a0 con 0x55e93d1df370 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93d246360 con 0x55e93d1e40c0 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.125+0000 7f8c8eee7b80 10 client.0 _ll_register_callbacks cb 0x7f8c6c097260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.126+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.126+0000 7f8c7939b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52856/0 (socket says 172.21.15.6:52856) 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.126+0000 7f8c7939b700 1 -- 172.21.15.6:0/923520363 learned_addr learned my addr 172.21.15.6:0/923520363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.126+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d245ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.126+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 -- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 msgr2=0x55e93d1ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 -- 172.21.15.6:0/923520363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 msgr2=0x55e93d1ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 -- 172.21.15.6:0/923520363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e93d0ee190 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c78b9a700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c7939b700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c79b9c700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d245ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55e93d1e0100 tx=0x7f8c6c086360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d60edf317cacfdd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.127+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c09ad30 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.128+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c6c084e40 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.128+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c70015070 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.128+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6c0a3a00 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 10 client.26269 fetch_fsmap learned FSMap version 22 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55e93cf55de0 con 0x55e93d1e40c0 2023-11-17T00:12:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70015070 con 0x55e93d1e40c0 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c6c0b0070 con 0x55e93d1e40c0 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 10 client.26269 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 20 client.26269 populate_metadata read hostname 'smithi006' 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d24d620 0x55e93d24fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 4 client.26269 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.129+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e93d0f6660 con 0x55e93d24d620 2023-11-17T00:12:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.130+0000 7f8c7939b700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d24d620 0x55e93d24fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.130+0000 7f8c5cff9700 10 client.26269 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.130+0000 7f8c7939b700 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d24d620 0x55e93d24fa00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca7c5dfa2c23864 server_cookie=99fde790f0e68c6a in_seq=0 out_seq=0 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.132+0000 7f8c5cff9700 1 -- 172.21.15.6:0/923520363 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55e93d0f6660 con 0x55e93d24d620 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.132+0000 7f8c5cff9700 10 client.26269 handle_command_reply: tid=0 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.132+0000 7f8c8eee7b80 1 client.26269 shutdown 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 msgr2=0x55e93d245ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d245ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55e93d1e0100 tx=0x7f8c6c086360 comp rx=0 tx=0).stop 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 shutdown_connections 2023-11-17T00:12:47.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e93d1e40c0 0x55e93d245ac0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e93d1df370 0x55e93d1ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e93d24d620 0x55e93d24fa00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 --2- 172.21.15.6:0/923520363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e93d1dfc80 0x55e93d1ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c5cff9700 0 client.26269 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 >> 172.21.15.6:0/923520363 conn(0x55e93d1cc360 msgr2=0x55e93d1e7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 shutdown_connections 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.133+0000 7f8c8eee7b80 1 -- 172.21.15.6:0/923520363 wait complete. 2023-11-17T00:12:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.134+0000 7f8c8eee7b80 20 client.26269 trim_cache size 0 max 16384 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (4639 inodes in the stack)", 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "92abc497-addd-4efc-ae92-66cf79369617": { 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "92abc497-addd-4efc-ae92-66cf79369617", 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:12:47.137 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.135+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 msgr2=0x7f8c600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.135+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 0x7f8c600a3870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400afd0 tx=0x7f8c6400a1f0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.135+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 msgr2=0x7f8c74198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.135+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009fd0 tx=0x7f8c6c00ce90 comp rx=0 tx=0).stop 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 shutdown_connections 2023-11-17T00:12:47.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8c600a13c0 0x7f8c600a3870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c7410ce50 0x7f8c74198730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c74073260 0x7f8c7419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 --2- 172.21.15.6:0/2047205283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740748a0 0x7f8c7419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 >> 172.21.15.6:0/2047205283 conn(0x7f8c74077530 msgr2=0x7f8c74104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 shutdown_connections 2023-11-17T00:12:47.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:12:47.136+0000 7f8c7b5ff700 1 -- 172.21.15.6:0/2047205283 wait complete. 2023-11-17T00:12:47.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:46 smithi006 ceph-mon[97840]: pgmap v119: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.0 MiB/s wr, 1.45k op/s 2023-11-17T00:12:47.166 INFO:tasks.cephfs.filesystem:scrub status for tag:92abc497-addd-4efc-ae92-66cf79369617 - {'path': '/', 'tag': '92abc497-addd-4efc-ae92-66cf79369617', 'options': 'recursive,force'} 2023-11-17T00:12:49.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:48 smithi087 ceph-mon[131182]: pgmap v120: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:48 smithi186 ceph-mon[131487]: pgmap v120: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:48 smithi006 ceph-mon[97840]: pgmap v120: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:50 smithi087 ceph-mon[131182]: pgmap v121: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:51.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:50 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:12:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:50 smithi186 ceph-mon[131487]: pgmap v121: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:50 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:12:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:50 smithi006 ceph-mon[97840]: pgmap v121: 129 pgs: 129 active+clean; 7.7 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.33k op/s 2023-11-17T00:12:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:50 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:12:52.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:51 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:12:52.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:51 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:12:52.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:51 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:12:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:52 smithi087 ceph-mon[131182]: pgmap v122: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 2.02k op/s 2023-11-17T00:12:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:52 smithi186 ceph-mon[131487]: pgmap v122: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 2.02k op/s 2023-11-17T00:12:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:52 smithi006 ceph-mon[97840]: pgmap v122: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 2.02k op/s 2023-11-17T00:12:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:54 smithi087 ceph-mon[131182]: pgmap v123: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 1.84k op/s 2023-11-17T00:13:07.644 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:54 smithi006 ceph-mon[97840]: pgmap v123: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 1.84k op/s 2023-11-17T00:13:07.644 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:56 smithi006 ceph-mon[97840]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 1.86k op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:56 smithi087 ceph-mon[131182]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 1.86k op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:12:58 smithi087 ceph-mon[131182]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:00 smithi087 ceph-mon[131182]: pgmap v126: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:02 smithi087 ceph-mon[131182]: pgmap v127: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 1.36k op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:04 smithi087 ceph-mon[131182]: pgmap v128: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 669 op/s 2023-11-17T00:13:07.646 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:06 smithi087 ceph-mon[131182]: pgmap v129: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:54 smithi186 ceph-mon[131487]: pgmap v123: 129 pgs: 129 active+clean; 19 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 1.84k op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:56 smithi186 ceph-mon[131487]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 1.86k op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:12:58 smithi186 ceph-mon[131487]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:00 smithi186 ceph-mon[131487]: pgmap v126: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:02 smithi186 ceph-mon[131487]: pgmap v127: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 1.36k op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:04 smithi186 ceph-mon[131487]: pgmap v128: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 669 op/s 2023-11-17T00:13:07.647 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:06 smithi186 ceph-mon[131487]: pgmap v129: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1958748006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 msgr2=0x7f7bf410cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1958748006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 0x7f7bf410cc30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009f90 tx=0x7f7bec00b330 comp rx=0 tx=0).stop 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1958748006 shutdown_connections 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1958748006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 0x7f7bf410cc30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1958748006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101440 0x7f7bf4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1958748006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101d50 0x7f7bf410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1958748006 >> 172.21.15.6:0/1958748006 conn(0x7f7bf40763a0 msgr2=0x7f7bf40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1958748006 shutdown_connections 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.599+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1958748006 wait complete. 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.601+0000 7f7bfb5ff700 1 Processor -- start 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.601+0000 7f7bfb5ff700 1 -- start start 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101440 0x7f7bf4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 0x7f7bf419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419d5e0 con 0x7f7bf410a810 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419d750 con 0x7f7bf4101d50 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bfb5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf419d910 con 0x7f7bf4101440 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59918/0 (socket says 172.21.15.6:59918) 2023-11-17T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 -- 172.21.15.6:0/1323828452 learned_addr learned my addr 172.21.15.6:0/1323828452 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf939b700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101440 0x7f7bf4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 -- 172.21.15.6:0/1323828452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101440 msgr2=0x7f7bf4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101440 0x7f7bf4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 -- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 msgr2=0x7f7bf419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 0x7f7bf419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.602+0000 7f7bf8b9a700 1 -- 172.21.15.6:0/1323828452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40095f0 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ec10 tx=0x7f7be400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bea7fc700 1 -- 172.21.15.6:0/1323828452 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be400cde0 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bea7fc700 1 -- 172.21.15.6:0/1323828452 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be40044e0 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bea7fc700 1 -- 172.21.15.6:0/1323828452 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4010490 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec009c40 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.603+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4112560 con 0x7f7bf4101d50 2023-11-17T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.608+0000 7f7bea7fc700 1 -- 172.21.15.6:0/1323828452 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7be4003780 con 0x7f7bf4101d50 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.610+0000 7f7c0efe4b80 1 Processor -- start 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.610+0000 7f7c0efe4b80 1 -- start start 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.610+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443ff6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c443ff6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443efc000 con 0x55c443ffcb90 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443f4e840 con 0x55c443e526b0 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443f47cd0 con 0x55c443ffc720 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bea7fc700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 0x7f7be00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bea7fc700 1 -- 172.21.15.6:0/1323828452 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7be4014070 con 0x7f7bf4101d50 2023-11-17T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bf9b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bf9b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59920/0 (socket says 172.21.15.6:59920) 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/1603427998 learned_addr learned my addr 172.21.15.6:0/1603427998 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c443ff6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.611+0000 7f7bf939b700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443ff6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/1603427998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 msgr2=0x55c443ff6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443ff6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/1603427998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 msgr2=0x55c443ff6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c443ff6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/1603427998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c443dc95d0 con 0x55c443e526b0 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c443ff6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 0x7f7be00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.612+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006070 tx=0x7f7bec009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c7e397293fb0ef server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.613+0000 7f7bdf7fe700 1 -- 172.21.15.6:0/1603427998 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00e050 con 0x55c443e526b0 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.613+0000 7f7bdf7fe700 1 -- 172.21.15.6:0/1603427998 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bec027900 con 0x55c443e526b0 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.613+0000 7f7bdf7fe700 1 -- 172.21.15.6:0/1603427998 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec01b770 con 0x55c443e526b0 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.613+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 0x7f7be00a3870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bec027000 tx=0x7f7bec028000 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1603427998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 msgr2=0x55c443ffc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bec006070 tx=0x7f7bec009640 comp rx=0 tx=0).stop 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1603427998 shutdown_connections 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c443ff6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c443ffc1e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1603427998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443ff6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1603427998 >> 172.21.15.6:0/1603427998 conn(0x55c443febf50 msgr2=0x55c443ff1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.614+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1603427998 shutdown_connections 2023-11-17T00:13:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.615+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1603427998 wait complete. 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.615+0000 7f7c0efe4b80 1 -- start start 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.616+0000 7f7c0efe4b80 1 Processor -- start 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443fffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c444000450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443ee9880 con 0x55c443ffcb90 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443ee9520 con 0x55c443e526b0 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443dcb8c0 con 0x55c443ffc720 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7c0efe4b80 10 client.0 _ll_register_callbacks cb 0x55c443f07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7bf939b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7bf939b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59924/0 (socket says 172.21.15.6:59924) 2023-11-17T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7bf939b700 1 -- 172.21.15.6:0/1623690564 learned_addr learned my addr 172.21.15.6:0/1623690564 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443fffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.618+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c444000450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 -- 172.21.15.6:0/1623690564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 msgr2=0x55c443fffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443fffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 -- 172.21.15.6:0/1623690564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 msgr2=0x55c444000450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c444000450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 -- 172.21.15.6:0/1623690564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf40ff180 con 0x55c443e526b0 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bf939b700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010590 tx=0x7f7bf0010950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02046a0ed7b8f64 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0014640 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7bf00147a0 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c443fe9710 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf0014960 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7c0efe4b80 10 client.26318 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c443f0e130 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c443fe9710 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.620+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bf0012c40 con 0x55c443e526b0 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 10 client.26318 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 20 client.26318 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44400fff0 0x55c444012400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 4 client.26318 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c443f0a040 con 0x55c44400fff0 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.621+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44400fff0 0x55c444012400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.622+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44400fff0 0x55c444012400 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9616b6c7039a71 server_cookie=cf53f97d66f4bbef in_seq=0 out_seq=0 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.622+0000 7f7bdcff9700 10 client.26318 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.622+0000 7f7bdcff9700 1 -- 172.21.15.6:0/1623690564 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c443f0a040 con 0x55c44400fff0 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7bdcff9700 10 client.26318 handle_command_reply: tid=0 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 client.26318 shutdown 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 msgr2=0x55c444002b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010590 tx=0x7f7bf0010950 comp rx=0 tx=0).stop 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 shutdown_connections 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44400fff0 0x55c444012400 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7bdcff9700 0 client.26318 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffcb90 0x55c444000450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444002b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/1623690564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffc720 0x55c443fffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 >> 172.21.15.6:0/1623690564 conn(0x55c443febf50 msgr2=0x55c443ff07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.623+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 shutdown_connections 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.624+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/1623690564 wait complete. 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.624+0000 7f7c0efe4b80 20 client.26318 trim_cache size 0 max 16384 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.647+0000 7f7c0efe4b80 1 Processor -- start 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 -- start start 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444007090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c443ff0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 0x55c444002630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c444000a50 con 0x55c443ffc720 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443ffaa10 con 0x55c443e526b0 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.648+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c443ffbdd0 con 0x55c443ffcb90 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444007090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf8b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c443ff0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf8b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c443ff0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37200/0 (socket says 172.21.15.6:37200) 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf8b9a700 1 -- 172.21.15.6:0/526229217 learned_addr learned my addr 172.21.15.6:0/526229217 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 0x55c444002630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 -- 172.21.15.6:0/526229217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 msgr2=0x55c444002630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 0x55c444002630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 -- 172.21.15.6:0/526229217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 msgr2=0x55c443ff0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c443ff0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf939b700 1 -- 172.21.15.6:0/526229217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c443f0de80 con 0x55c443e526b0 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.649+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 0x55c444002630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.650+0000 7f7bf939b700 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444007090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010560 tx=0x7f7bf0006af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39abbbaa8cad3d3d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.650+0000 7f7bdffff700 1 -- 172.21.15.6:0/526229217 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002c070 con 0x55c443e526b0 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.650+0000 7f7bdffff700 1 -- 172.21.15.6:0/526229217 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf000b480 con 0x55c443e526b0 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.650+0000 7f7bdffff700 1 -- 172.21.15.6:0/526229217 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0013db0 con 0x55c443e526b0 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/526229217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 msgr2=0x55c444007090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444007090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0010560 tx=0x7f7bf0006af0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/526229217 shutdown_connections 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443ffcb90 0x55c444002630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443e526b0 0x55c444007090 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/526229217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c443ff0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/526229217 >> 172.21.15.6:0/526229217 conn(0x55c443febf50 msgr2=0x55c443ff9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/526229217 shutdown_connections 2023-11-17T00:13:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/526229217 wait complete. 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.651+0000 7f7c0efe4b80 1 -- start start 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.652+0000 7f7c0efe4b80 1 Processor -- start 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 0x55c444006850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c444006d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c444003b70 con 0x55c443ffc720 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c444003ce0 con 0x55c443ffcb90 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c444003ea0 con 0x55c443e526b0 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7c0efe4b80 10 client.0 _ll_register_callbacks cb 0x55c443f07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59942/0 (socket says 172.21.15.6:59942) 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/2800575806 learned_addr learned my addr 172.21.15.6:0/2800575806 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf939b700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 0x55c444006850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf8b9a700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c444006d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/2800575806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 msgr2=0x55c444006850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 0x55c444006850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/2800575806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 msgr2=0x55c444006d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c444006d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.653+0000 7f7bf9b9c700 1 -- 172.21.15.6:0/2800575806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec02b030 con 0x55c443ffcb90 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7bf9b9c700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00d010 tx=0x7f7bec02c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202d4b44fa9c739a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7bf939b700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 0x55c444006850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec027640 con 0x55c443ffcb90 2023-11-17T00:13:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7bec01b430 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.654+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bec027900 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bec0277a0 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 10 client.26324 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c443f0e130 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bec027900 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bec00b9f0 con 0x55c443ffcb90 2023-11-17T00:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 10 client.26324 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 20 client.26324 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44406d280 0x55c44406f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 4 client.26324 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.655+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c443f16600 con 0x55c44406d280 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.656+0000 7f7bf939b700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44406d280 0x55c44406f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.656+0000 7f7bdcff9700 10 client.26324 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:12:57.656+0000 7f7bf939b700 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44406d280 0x55c44406f690 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d55c843205fce8 server_cookie=12c42ac7ce990ab7 in_seq=0 out_seq=0 2023-11-17T00:13:07.798 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:12:58 smithi006 ceph-mon[97840]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.798 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:00 smithi006 ceph-mon[97840]: pgmap v126: 129 pgs: 129 active+clean; 32 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 714 op/s 2023-11-17T00:13:07.798 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:02 smithi006 ceph-mon[97840]: pgmap v127: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 1.36k op/s 2023-11-17T00:13:07.798 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:04 smithi006 ceph-mon[97840]: pgmap v128: 129 pgs: 129 active+clean; 42 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 669 op/s 2023-11-17T00:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.322+0000 7f7bdcff9700 1 -- 172.21.15.6:0/2800575806 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c443f16600 con 0x55c44406d280 2023-11-17T00:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.322+0000 7f7bdcff9700 10 client.26324 handle_command_reply: tid=0 2023-11-17T00:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.322+0000 7f7c0efe4b80 1 client.26324 shutdown 2023-11-17T00:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 msgr2=0x55c444003630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00d010 tx=0x7f7bec02c000 comp rx=0 tx=0).stop 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 shutdown_connections 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c443ffcb90 0x55c444003630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c443e526b0 0x55c444006850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c44406d280 0x55c44406f690 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 --2- 172.21.15.6:0/2800575806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c443ffc720 0x55c444006d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 >> 172.21.15.6:0/2800575806 conn(0x55c443febf50 msgr2=0x55c443ffae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7bdcff9700 0 client.26324 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 shutdown_connections 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.323+0000 7f7c0efe4b80 1 -- 172.21.15.6:0/2800575806 wait complete. 2023-11-17T00:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.324+0000 7f7c0efe4b80 20 client.26324 trim_cache size 0 max 16384 2023-11-17T00:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 msgr2=0x7f7be00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 0x7f7be00a3870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bec027000 tx=0x7f7bec028000 comp rx=0 tx=0).stop 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 msgr2=0x7f7bf4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ec10 tx=0x7f7be400c2d0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 shutdown_connections 2023-11-17T00:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7be00a13c0 0x7f7be00a3870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7bf410a810 0x7f7bf419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7bf4101440 0x7f7bf4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 --2- 172.21.15.6:0/1323828452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bf4101d50 0x7f7bf4198b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 >> 172.21.15.6:0/1323828452 conn(0x7f7bf40763a0 msgr2=0x7f7bf41021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 shutdown_connections 2023-11-17T00:13:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:06.325+0000 7f7bfb5ff700 1 -- 172.21.15.6:0/1323828452 wait complete. 2023-11-17T00:13:07.827 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:06 smithi006 ceph-mon[97840]: pgmap v129: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2023-11-17T00:13:09.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:08 smithi006 ceph-mon[97840]: pgmap v130: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:09.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:08 smithi087 ceph-mon[131182]: pgmap v130: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:09.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:08 smithi186 ceph-mon[131487]: pgmap v130: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:10 smithi006 ceph-mon[97840]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:10 smithi087 ceph-mon[131182]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:10 smithi186 ceph-mon[131487]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 3.66k op/s 2023-11-17T00:13:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:12 smithi006 ceph-mon[97840]: pgmap v132: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 3.85k op/s 2023-11-17T00:13:13.356 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:12 smithi087 ceph-mon[131182]: pgmap v132: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 3.85k op/s 2023-11-17T00:13:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:12 smithi186 ceph-mon[131487]: pgmap v132: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 3.85k op/s 2023-11-17T00:13:15.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:14 smithi006 ceph-mon[97840]: pgmap v133: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 870 KiB/s wr, 3.20k op/s 2023-11-17T00:13:15.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:14 smithi087 ceph-mon[131182]: pgmap v133: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 870 KiB/s wr, 3.20k op/s 2023-11-17T00:13:15.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:14 smithi186 ceph-mon[131487]: pgmap v133: 129 pgs: 129 active+clean; 11 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 870 KiB/s wr, 3.20k op/s 2023-11-17T00:13:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:15 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:13:16.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:15 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:13:16.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:15 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:13:17.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:13:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:16 smithi006 ceph-mon[97840]: pgmap v134: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 3.21k op/s 2023-11-17T00:13:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:16 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:13:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:16 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:13:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:16 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:13:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:16 smithi087 ceph-mon[131182]: pgmap v134: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 3.21k op/s 2023-11-17T00:13:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:16 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:13:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:16 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:13:17.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:16 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:13:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:16 smithi186 ceph-mon[131487]: pgmap v134: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 3.21k op/s 2023-11-17T00:13:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:16 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:13:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:16 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:13:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:16 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:13:17.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 -- 172.21.15.6:0/3975126571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 msgr2=0x7f8c04101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 --2- 172.21.15.6:0/3975126571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c04101020 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40099c0 tx=0x7f8bf4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 -- 172.21.15.6:0/3975126571 shutdown_connections 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 --2- 172.21.15.6:0/3975126571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c04101020 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 --2- 172.21.15.6:0/3975126571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c04105560 0x7f8c0410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 --2- 172.21.15.6:0/3975126571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c041015f0 0x7f8c04104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 -- 172.21.15.6:0/3975126571 >> 172.21.15.6:0/3975126571 conn(0x7f8c04077500 msgr2=0x7f8c04077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 -- 172.21.15.6:0/3975126571 shutdown_connections 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.382+0000 7f8c08bff700 1 -- 172.21.15.6:0/3975126571 wait complete. 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.383+0000 7f8c08bff700 1 Processor -- start 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 -- start start 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c041985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 0x7f8c04198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 0x7f8c0419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419d5a0 con 0x7f8c04100c50 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419d710 con 0x7f8c041015f0 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c08bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419d8d0 con 0x7f8c04105560 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c041985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 0x7f8c04198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c041985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39792/0 (socket says 172.21.15.6:39792) 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c0259c700 1 -- 172.21.15.6:0/910686275 learned_addr learned my addr 172.21.15.6:0/910686275 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.384+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 0x7f8c0419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 -- 172.21.15.6:0/910686275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 msgr2=0x7f8c0419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 0x7f8c0419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 -- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 msgr2=0x7f8c041985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c041985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 -- 172.21.15.6:0/910686275 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40095f0 con 0x7f8c041015f0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 0x7f8c0419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 0x7f8c04198b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c041019c0 tx=0x7f8bf800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8bf37fe700 1 -- 172.21.15.6:0/910686275 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800cbd0 con 0x7f8c041015f0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c04110360 con 0x7f8c041015f0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8bf37fe700 1 -- 172.21.15.6:0/910686275 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf800ee80 con 0x7f8c041015f0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8bf37fe700 1 -- 172.21.15.6:0/910686275 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8018680 con 0x7f8c041015f0 2023-11-17T00:13:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.385+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c041101d0 con 0x7f8c041015f0 2023-11-17T00:13:17.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.388+0000 7f8bf37fe700 1 -- 172.21.15.6:0/910686275 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8bf8010430 con 0x7f8c041015f0 2023-11-17T00:13:17.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 Processor -- start 2023-11-17T00:13:17.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 -- start start 2023-11-17T00:13:17.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8bf37fe700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 0x7f8bec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8bf37fe700 1 -- 172.21.15.6:0/910686275 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bf8014070 con 0x7f8c041015f0 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 0x7f8bec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e2b9060 con 0x56339e3b4470 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e30b8a0 con 0x56339e3b88b0 2023-11-17T00:13:17.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.390+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e304d30 con 0x56339e3b3b60 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c02d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c02d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58714/0 (socket says 172.21.15.6:58714) 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c02d9d700 1 -- 172.21.15.6:0/4227797161 learned_addr learned my addr 172.21.15.6:0/4227797161 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4227797161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 msgr2=0x56339e3b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4227797161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 msgr2=0x56339e3b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4227797161 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339e186630 con 0x56339e3b4470 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c0259c700 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.391+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 0x7f8bec0a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007cf0 tx=0x7f8bfc009300 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3b8370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80ca640 tx=0x7f8bf8096430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13aaac9005da035 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8bf0ff9700 1 -- 172.21.15.6:0/4227797161 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8099fc0 con 0x56339e3b4470 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8bf0ff9700 1 -- 172.21.15.6:0/4227797161 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf809dff0 con 0x56339e3b4470 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8bf0ff9700 1 -- 172.21.15.6:0/4227797161 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80996f0 con 0x56339e3b4470 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4227797161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 msgr2=0x56339e3b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3b8370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80ca640 tx=0x7f8bf8096430 comp rx=0 tx=0).stop 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4227797161 shutdown_connections 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4227797161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3b8370 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.392+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4227797161 >> 172.21.15.6:0/4227797161 conn(0x56339e3a9360 msgr2=0x56339e3aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.393+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4227797161 shutdown_connections 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.393+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4227797161 wait complete. 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.393+0000 7f8c1c54db80 1 -- start start 2023-11-17T00:13:17.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.393+0000 7f8c1c54db80 1 Processor -- start 2023-11-17T00:13:17.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e2a68e0 con 0x56339e3b4470 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e2a6580 con 0x56339e3b88b0 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e188920 con 0x56339e3b3b60 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c1c54db80 10 client.0 _ll_register_callbacks cb 0x56339e2c4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c02d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39808/0 (socket says 172.21.15.6:39808) 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c01d9b700 1 -- 172.21.15.6:0/2030205069 learned_addr learned my addr 172.21.15.6:0/2030205069 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.394+0000 7f8c0259c700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 -- 172.21.15.6:0/2030205069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 msgr2=0x56339e3c00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 -- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 msgr2=0x56339e3bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 -- 172.21.15.6:0/2030205069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c041101d0 con 0x56339e3b4470 2023-11-17T00:13:17.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80cabc0 tx=0x7f8bf809a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ece6ab0884d487 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8be9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80a2410 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bf8099fc0 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339e3a6b20 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf8094de0 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c1c54db80 10 client.16218 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56339e2cb190 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.395+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56339e3a6b20 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bf80ca140 con 0x56339e3b4470 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 10 client.16218 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 20 client.16218 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:17.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e3cd350 0x56339e3cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 4 client.16218 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:17.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56339e2c70a0 con 0x56339e3cd350 2023-11-17T00:13:17.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c0259c700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e3cd350 0x56339e3cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8c0259c700 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e3cd350 0x56339e3cf730 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e39bb0c07683a3e server_cookie=c88887d154fa9399 in_seq=0 out_seq=0 2023-11-17T00:13:17.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.396+0000 7f8be9ffb700 10 client.16218 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.398+0000 7f8be9ffb700 1 -- 172.21.15.6:0/2030205069 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56339e2c70a0 con 0x56339e3cd350 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.398+0000 7f8be9ffb700 10 client.16218 handle_command_reply: tid=0 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.398+0000 7f8c1c54db80 1 client.16218 shutdown 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 msgr2=0x56339e3bd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80cabc0 tx=0x7f8bf809a450 comp rx=0 tx=0).stop 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 shutdown_connections 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b88b0 0x56339e3bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e3cd350 0x56339e3cf730 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/2030205069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b4470 0x56339e3bd2c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 >> 172.21.15.6:0/2030205069 conn(0x56339e3a9360 msgr2=0x56339e3adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8be9ffb700 0 client.16218 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 shutdown_connections 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 1 -- 172.21.15.6:0/2030205069 wait complete. 2023-11-17T00:13:17.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.399+0000 7f8c1c54db80 20 client.16218 trim_cache size 0 max 16384 2023-11-17T00:13:17.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 Processor -- start 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 -- start start 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b88b0 0x56339e3bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3a8ee0 con 0x56339e3b88b0 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3cfcf0 con 0x56339e3b4470 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3b73f0 con 0x56339e3b3b60 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58728/0 (socket says 172.21.15.6:58728) 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c01d9b700 1 -- 172.21.15.6:0/1514534654 learned_addr learned my addr 172.21.15.6:0/1514534654 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b88b0 0x56339e3bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.417+0000 7f8c0259c700 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 -- 172.21.15.6:0/1514534654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 msgr2=0x56339e3c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 -- 172.21.15.6:0/1514534654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b88b0 msgr2=0x56339e3bfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b88b0 0x56339e3bfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 -- 172.21.15.6:0/1514534654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339e2caee0 con 0x56339e3b4470 2023-11-17T00:13:17.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800f010 tx=0x7f8bf8010a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740cca1b171ffc16 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8bf17fa700 1 -- 172.21.15.6:0/1514534654 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf809a4c0 con 0x56339e3b4470 2023-11-17T00:13:17.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8bf17fa700 1 -- 172.21.15.6:0/1514534654 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf80ca4f0 con 0x56339e3b4470 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.418+0000 7f8bf17fa700 1 -- 172.21.15.6:0/1514534654 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80ca2c0 con 0x56339e3b4470 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- 172.21.15.6:0/1514534654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 msgr2=0x56339e3c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800f010 tx=0x7f8bf8010a50 comp rx=0 tx=0).stop 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- 172.21.15.6:0/1514534654 shutdown_connections 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b88b0 0x56339e3bfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b3b60 0x56339e3c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/1514534654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c4940 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- 172.21.15.6:0/1514534654 >> 172.21.15.6:0/1514534654 conn(0x56339e3a9360 msgr2=0x56339e3bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- 172.21.15.6:0/1514534654 shutdown_connections 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- 172.21.15.6:0/1514534654 wait complete. 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 -- start start 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.419+0000 7f8c1c54db80 1 Processor -- start 2023-11-17T00:13:17.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b3b60 0x56339e3c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 0x56339e3c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3c44d0 con 0x56339e3b3b60 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3c4640 con 0x56339e3b4470 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e3c4800 con 0x56339e3b88b0 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c1c54db80 10 client.0 _ll_register_callbacks cb 0x7f8bfc007840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b3b60 0x56339e3c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c01d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58736/0 (socket says 172.21.15.6:58736) 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4127411852 learned_addr learned my addr 172.21.15.6:0/4127411852 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.420+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 0x56339e3c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4127411852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 msgr2=0x56339e3c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 0x56339e3c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4127411852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b3b60 msgr2=0x56339e3c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b3b60 0x56339e3c0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 -- 172.21.15.6:0/4127411852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56339e2cb190 con 0x56339e3b4470 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c02d9d700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 0x56339e3c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c01d9b700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8098310 tx=0x7f8bf8010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3509345f0bede8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8be9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8099fc0 con 0x56339e3b4470 2023-11-17T00:13:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bf809dff0 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bf80a1f80 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf80ca140 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c1c54db80 10 client.26342 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56339e11bc60 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.421+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf809dff0 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bf8099020 con 0x56339e3b4470 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 10 client.26342 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 20 client.26342 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e421d00 0x56339e4240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 4 client.26342 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56339e2d3660 con 0x56339e421d00 2023-11-17T00:13:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c0259c700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e421d00 0x56339e4240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8c0259c700 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e421d00 0x56339e4240e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0549781c1fe13a4 server_cookie=f13b60921d9869c8 in_seq=0 out_seq=0 2023-11-17T00:13:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.422+0000 7f8be9ffb700 10 client.26342 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.424+0000 7f8be9ffb700 1 -- 172.21.15.6:0/4127411852 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56339e2d3660 con 0x56339e421d00 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.424+0000 7f8be9ffb700 10 client.26342 handle_command_reply: tid=0 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.424+0000 7f8c1c54db80 1 client.26342 shutdown 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 msgr2=0x56339e3c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8098310 tx=0x7f8bf8010c50 comp rx=0 tx=0).stop 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 shutdown_connections 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56339e421d00 0x56339e4240e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56339e3b3b60 0x56339e3c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8be9ffb700 0 client.26342 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56339e3b88b0 0x56339e3c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 --2- 172.21.15.6:0/4127411852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56339e3b4470 0x56339e3c1050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 >> 172.21.15.6:0/4127411852 conn(0x56339e3a9360 msgr2=0x56339e3bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 shutdown_connections 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 1 -- 172.21.15.6:0/4127411852 wait complete. 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c1c54db80 20 client.26342 trim_cache size 0 max 16384 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:13:17.446 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:13:17.447 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 msgr2=0x7f8bec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 0x7f8bec0a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007cf0 tx=0x7f8bfc009300 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 msgr2=0x7f8c04198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.425+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 0x7f8c04198b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c041019c0 tx=0x7f8bf800eca0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 shutdown_connections 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8bec0a12c0 0x7f8bec0a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c04100c50 0x7f8c041985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c04105560 0x7f8c0419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 --2- 172.21.15.6:0/910686275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c041015f0 0x7f8c04198b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 >> 172.21.15.6:0/910686275 conn(0x7f8c04077500 msgr2=0x7f8c040fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 shutdown_connections 2023-11-17T00:13:17.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.426+0000 7f8c08bff700 1 -- 172.21.15.6:0/910686275 wait complete. 2023-11-17T00:13:17.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:13:17.449 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:13:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 -- 172.21.15.6:0/3036808924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c105620 msgr2=0x7fb29c10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/3036808924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c105620 0x7fb29c10d1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b0d0 tx=0x7fb29400b3e0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 -- 172.21.15.6:0/3036808924 shutdown_connections 2023-11-17T00:13:17.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/3036808924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c105620 0x7fb29c10d1a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/3036808924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 0x7fb29c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/3036808924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c101660 0x7fb29c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 -- 172.21.15.6:0/3036808924 >> 172.21.15.6:0/3036808924 conn(0x7fb29c0774d0 msgr2=0x7fb29c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 -- 172.21.15.6:0/3036808924 shutdown_connections 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.664+0000 7fb2a21ff700 1 -- 172.21.15.6:0/3036808924 wait complete. 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 Processor -- start 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 -- start start 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 0x7fb29c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c073b60 con 0x7fb29c105620 2023-11-17T00:13:17.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1a4cb0 con 0x7fb29c101660 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.665+0000 7fb2a21ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1a4df0 con 0x7fb29c100cc0 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb293fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb293fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58750/0 (socket says 172.21.15.6:58750) 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb293fff700 1 -- 172.21.15.6:0/1949311606 learned_addr learned my addr 172.21.15.6:0/1949311606 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39834/0 (socket says 172.21.15.6:39834) 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29b7fe700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 0x7fb29c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 -- 172.21.15.6:0/1949311606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 msgr2=0x7fb29c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 0x7fb29c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 -- 172.21.15.6:0/1949311606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 msgr2=0x7fb29c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 -- 172.21.15.6:0/1949311606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009d00 con 0x7fb29c105620 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb293fff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.666+0000 7fb29bfff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb294009fd0 tx=0x7fb2940077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.667+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400e040 con 0x7fb29c105620 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.667+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c1103e0 con 0x7fb29c105620 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.667+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29400bcb0 con 0x7fb29c105620 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.667+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294005840 con 0x7fb29c105620 2023-11-17T00:13:17.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.667+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c110250 con 0x7fb29c105620 2023-11-17T00:13:17.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.668+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c110570 con 0x7fb29c105620 2023-11-17T00:13:17.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.670+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb294004a50 con 0x7fb29c105620 2023-11-17T00:13:17.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.672+0000 7fb2997fa700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 0x7fb27c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:17.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.672+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb2940ce170 con 0x7fb29c105620 2023-11-17T00:13:17.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.672+0000 7fb29b7fe700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 0x7fb27c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:17.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.673+0000 7fb29b7fe700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 0x7fb27c0a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009c60 tx=0x7fb28c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:17.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.676+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb294097260 con 0x7fb29c105620 2023-11-17T00:13:17.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.934+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb29c067af0 con 0x7fb29c105620 2023-11-17T00:13:17.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.935+0000 7fb2997fa700 1 -- 172.21.15.6:0/1949311606 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fb29409a880 con 0x7fb29c105620 2023-11-17T00:13:17.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:13:17.936 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:13:17.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.938+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 msgr2=0x7fb27c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.938+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 0x7fb27c0a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009c60 tx=0x7fb28c009260 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.938+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 msgr2=0x7fb29c1a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.938+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb294009fd0 tx=0x7fb2940077f0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 shutdown_connections 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fb27c0a12c0 0x7fb27c0a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb29c105620 0x7fb29c1a4770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c100cc0 0x7fb29c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 --2- 172.21.15.6:0/1949311606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb29c101660 0x7fb29c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 >> 172.21.15.6:0/1949311606 conn(0x7fb29c0774d0 msgr2=0x7fb29c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 shutdown_connections 2023-11-17T00:13:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:17.939+0000 7fb2a21ff700 1 -- 172.21.15.6:0/1949311606 wait complete. 2023-11-17T00:13:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:13:17.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:13:18.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 -- 172.21.15.6:0/4105161084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4105560 msgr2=0x7f5ba410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/4105161084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4105560 0x7f5ba410d130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b0d0 tx=0x7f5b9c00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 -- 172.21.15.6:0/4105161084 shutdown_connections 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/4105161084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4105560 0x7f5ba410d130 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/4105161084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba4100c50 0x7f5ba4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/4105161084 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba41015f0 0x7f5ba4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 -- 172.21.15.6:0/4105161084 >> 172.21.15.6:0/4105161084 conn(0x7f5ba4077500 msgr2=0x7f5ba4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 -- 172.21.15.6:0/4105161084 shutdown_connections 2023-11-17T00:13:18.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.171+0000 7f5ba99ff700 1 -- 172.21.15.6:0/4105161084 wait complete. 2023-11-17T00:13:18.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.172+0000 7f5ba99ff700 1 Processor -- start 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.172+0000 7f5ba99ff700 1 -- start start 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba41015f0 0x7f5ba41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 0x7f5ba419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419b470 con 0x7f5ba4105560 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419b5e0 con 0x7f5ba4100c50 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba99ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419b7a0 con 0x7f5ba41015f0 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58768/0 (socket says 172.21.15.6:58768) 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/486777253 learned_addr learned my addr 172.21.15.6:0/486777253 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba41015f0 0x7f5ba41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 0x7f5ba419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/486777253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba41015f0 msgr2=0x7f5ba41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba41015f0 0x7f5ba41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 msgr2=0x7f5ba419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 0x7f5ba419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/486777253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c009d00 con 0x7f5ba4100c50 2023-11-17T00:13:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.173+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 0x7f5ba419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400e9a0 tx=0x7f5b9400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba89fd700 1 -- 172.21.15.6:0/486777253 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94009790 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba89fd700 1 -- 172.21.15.6:0/486777253 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9400cbd0 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba89fd700 1 -- 172.21.15.6:0/486777253 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94018680 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4110360 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.174+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba41101d0 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.178+0000 7f5ba89fd700 1 -- 172.21.15.6:0/486777253 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5b9400cd40 con 0x7f5ba4100c50 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.178+0000 7f5bbd229b80 1 Processor -- start 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 -- start start 2023-11-17T00:13:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e451006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e451007110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450f03060 con 0x55e451006cc0 2023-11-17T00:13:18.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450f558a0 con 0x55e451001f70 2023-11-17T00:13:18.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.179+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450f4ed30 con 0x55e451002880 2023-11-17T00:13:18.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58780/0 (socket says 172.21.15.6:58780) 2023-11-17T00:13:18.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 -- 172.21.15.6:0/1467027128 learned_addr learned my addr 172.21.15.6:0/1467027128 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:18.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e451006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e451007110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba89fd700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 0x7f5b8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 -- 172.21.15.6:0/1467027128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 msgr2=0x55e451006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e451006780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 -- 172.21.15.6:0/1467027128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 msgr2=0x55e451007110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e451007110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 -- 172.21.15.6:0/1467027128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e450dd0630 con 0x55e451001f70 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba89fd700 1 -- 172.21.15.6:0/486777253 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b94014070 con 0x7f5ba4100c50 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 0x7f5b8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.180+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009380 tx=0x7f5b9c0093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91611d5e381bf7d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5b89ffb700 1 -- 172.21.15.6:0/1467027128 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c00e040 con 0x55e451001f70 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5b89ffb700 1 -- 172.21.15.6:0/1467027128 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9c022070 con 0x55e451001f70 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5b89ffb700 1 -- 172.21.15.6:0/1467027128 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0056f0 con 0x55e451001f70 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5bbd229b80 1 -- 172.21.15.6:0/1467027128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 msgr2=0x55e451002340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 0x7f5b8c0a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c029000 tx=0x7f5b9c026000 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.181+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009380 tx=0x7f5b9c0093b0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 -- 172.21.15.6:0/1467027128 shutdown_connections 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e451007110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e451002340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/1467027128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e451006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 -- 172.21.15.6:0/1467027128 >> 172.21.15.6:0/1467027128 conn(0x55e450ff3360 msgr2=0x55e450ff8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 -- 172.21.15.6:0/1467027128 shutdown_connections 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 -- 172.21.15.6:0/1467027128 wait complete. 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.182+0000 7f5bbd229b80 1 -- start start 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 Processor -- start 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e45100b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e45100bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450ef08e0 con 0x55e451006cc0 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450ef0580 con 0x55e451001f70 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.183+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450dd2920 con 0x55e451002880 2023-11-17T00:13:18.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5bbd229b80 10 client.0 _ll_register_callbacks cb 0x55e450f0e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58794/0 (socket says 172.21.15.6:58794) 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/2979020289 learned_addr learned my addr 172.21.15.6:0/2979020289 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e45100bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e45100b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/2979020289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 msgr2=0x55e45100b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e45100b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/2979020289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 msgr2=0x55e45100bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e45100bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 -- 172.21.15.6:0/2979020289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba410d940 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b94096be0 tx=0x7f5b9400efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce58224e3f0b38b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5b82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9409a5d0 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.184+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b9409a170 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e450ff0b20 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b94099ad0 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 10 client.26360 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55e450f15190 con 0x55e451001f70 2023-11-17T00:13:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9407d080 con 0x55e451001f70 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b940a2130 con 0x55e451001f70 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 20 client.26360 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e45101bc80 0x55e45101e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 4 client.26360 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e450f110a0 con 0x55e45101bc80 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.185+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e45101bc80 0x55e45101e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.186+0000 7f5b82ffd700 10 client.26360 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.186+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e45101bc80 0x55e45101e090 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7067d4599faee9ea server_cookie=b2cddb7e06d5b105 in_seq=0 out_seq=0 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2979020289 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e450f110a0 con 0x55e45101bc80 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5b82ffd700 10 client.26360 handle_command_reply: tid=0 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 client.26360 shutdown 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 msgr2=0x55e45100e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b94096be0 tx=0x7f5b9400efa0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 shutdown_connections 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e45101bc80 0x55e45101e090 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451006cc0 0x55e45100bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451001f70 0x55e45100e3d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5b82ffd700 0 client.26360 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:18.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2979020289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451002880 0x55e45100b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 >> 172.21.15.6:0/2979020289 conn(0x55e450ff3360 msgr2=0x55e451008170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 shutdown_connections 2023-11-17T00:13:18.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.188+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2979020289 wait complete. 2023-11-17T00:13:18.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.189+0000 7f5bbd229b80 20 client.26360 trim_cache size 0 max 16384 2023-11-17T00:13:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.202+0000 7f5bbd229b80 1 Processor -- start 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.202+0000 7f5bbd229b80 1 -- start start 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451012db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e450ff8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450ffa430 con 0x55e451002880 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e450ffa1c0 con 0x55e451006cc0 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e451009bb0 con 0x55e451001f70 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451012db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39874/0 (socket says 172.21.15.6:39874) 2023-11-17T00:13:18.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 -- 172.21.15.6:0/4260022280 learned_addr learned my addr 172.21.15.6:0/4260022280 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:18.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e450ff8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 -- 172.21.15.6:0/4260022280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 msgr2=0x55e451012db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451012db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 -- 172.21.15.6:0/4260022280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 msgr2=0x55e450ff8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e450ff8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba27fc700 1 -- 172.21.15.6:0/4260022280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e451007360 con 0x55e451002880 2023-11-17T00:13:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451012db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.203+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e450ff8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:18.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4197aa0 tx=0x7f5b98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb8fb4d8a13fce7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5b8a7fc700 1 -- 172.21.15.6:0/4260022280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98016030 con 0x55e451002880 2023-11-17T00:13:18.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5b8a7fc700 1 -- 172.21.15.6:0/4260022280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b98017040 con 0x55e451002880 2023-11-17T00:13:18.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5b8a7fc700 1 -- 172.21.15.6:0/4260022280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98003bb0 con 0x55e451002880 2023-11-17T00:13:18.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5bbd229b80 1 -- 172.21.15.6:0/4260022280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 msgr2=0x55e45100df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4197aa0 tx=0x7f5b98009260 comp rx=0 tx=0).stop 2023-11-17T00:13:18.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.204+0000 7f5bbd229b80 1 -- 172.21.15.6:0/4260022280 shutdown_connections 2023-11-17T00:13:18.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e450ff8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451012db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/4260022280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100df70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 -- 172.21.15.6:0/4260022280 >> 172.21.15.6:0/4260022280 conn(0x55e450ff3360 msgr2=0x55e451004d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 -- 172.21.15.6:0/4260022280 shutdown_connections 2023-11-17T00:13:18.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 -- 172.21.15.6:0/4260022280 wait complete. 2023-11-17T00:13:18.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 -- start start 2023-11-17T00:13:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.205+0000 7f5bbd229b80 1 Processor -- start 2023-11-17T00:13:18.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451009e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e45100af20 con 0x55e451002880 2023-11-17T00:13:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e45100b090 con 0x55e451006cc0 2023-11-17T00:13:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e45100b250 con 0x55e451001f70 2023-11-17T00:13:18.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5bbd229b80 10 client.0 _ll_register_callbacks cb 0x55e450f0e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:18.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5ba27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5ba37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:58814/0 (socket says 172.21.15.6:58814) 2023-11-17T00:13:18.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5ba37fe700 1 -- 172.21.15.6:0/2417554447 learned_addr learned my addr 172.21.15.6:0/2417554447 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:18.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.206+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451009e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 -- 172.21.15.6:0/2417554447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 msgr2=0x55e451009e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451009e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 -- 172.21.15.6:0/2417554447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 msgr2=0x55e45100a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:18.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 -- 172.21.15.6:0/2417554447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c02d040 con 0x55e451006cc0 2023-11-17T00:13:18.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba27fc700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:18.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5ba37fe700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b770 tx=0x7f5b9c00baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1dec91b6b51f817 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:18.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5b82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:18.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c00e040 con 0x55e451006cc0 2023-11-17T00:13:18.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b9c005590 con 0x55e451006cc0 2023-11-17T00:13:18.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b9c022070 con 0x55e451006cc0 2023-11-17T00:13:18.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.207+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b9c005750 con 0x55e451006cc0 2023-11-17T00:13:18.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 10 client.26366 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:18.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f5b980044d0 con 0x55e451006cc0 2023-11-17T00:13:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9c022070 con 0x55e451006cc0 2023-11-17T00:13:18.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b9c033a10 con 0x55e451006cc0 2023-11-17T00:13:18.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 10 client.26366 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:18.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:18.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 20 client.26366 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:18.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e4510705b0 0x55e4510729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:18.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 4 client.26366 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:13:18.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e450f1d660 con 0x55e4510705b0 2023-11-17T00:13:18.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.208+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e4510705b0 0x55e4510729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:18.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.209+0000 7f5ba2ffd700 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e4510705b0 0x55e4510729c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df9d38e4179d590 server_cookie=d1036191a5eb0f71 in_seq=0 out_seq=0 2023-11-17T00:13:18.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.209+0000 7f5b82ffd700 10 client.26366 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:18.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5b82ffd700 1 -- 172.21.15.6:0/2417554447 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e450f1d660 con 0x55e4510705b0 2023-11-17T00:13:18.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5b82ffd700 10 client.26366 handle_command_reply: tid=0 2023-11-17T00:13:18.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 client.26366 shutdown 2023-11-17T00:13:18.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 msgr2=0x55e45100a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00b770 tx=0x7f5b9c00baa0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 shutdown_connections 2023-11-17T00:13:18.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e451006cc0 0x55e45100a880 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e451001f70 0x55e451009e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55e4510705b0 0x55e4510729c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 --2- 172.21.15.6:0/2417554447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e451002880 0x55e45100a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5b82ffd700 0 client.26366 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 >> 172.21.15.6:0/2417554447 conn(0x55e450ff3360 msgr2=0x55e451008980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 shutdown_connections 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 1 -- 172.21.15.6:0/2417554447 wait complete. 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.211+0000 7f5bbd229b80 20 client.26366 trim_cache size 0 max 16384 2023-11-17T00:13:18.336 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:13:18.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 msgr2=0x7f5b8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 0x7f5b8c0a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c029000 tx=0x7f5b9c026000 comp rx=0 tx=0).stop 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 msgr2=0x7f5ba4196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400e9a0 tx=0x7f5b9400ed60 comp rx=0 tx=0).stop 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 shutdown_connections 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5b8c0a12c0 0x7f5b8c0a3770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ba4105560 0x7f5ba419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba4100c50 0x7f5ba4196490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 --2- 172.21.15.6:0/486777253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ba41015f0 0x7f5ba41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:18.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 >> 172.21.15.6:0/486777253 conn(0x7f5ba4077500 msgr2=0x7f5ba4103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:18.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 shutdown_connections 2023-11-17T00:13:18.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:18.212+0000 7f5ba99ff700 1 -- 172.21.15.6:0/486777253 wait complete. 2023-11-17T00:13:18.339 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:17 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:13:18.339 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:17 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:13:18.339 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:17 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:13:18.339 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:17 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:13:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:17 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:13:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:17 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:13:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:17 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:13:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:17 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:13:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:17 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:13:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:17 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:13:18.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:17 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:13:18.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:17 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:13:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:18 smithi006 ceph-mon[97840]: pgmap v135: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1949311606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:13:19.341 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:13:19.342 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:13:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:18 smithi087 ceph-mon[131182]: pgmap v135: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:19.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:18 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1949311606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:13:19.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:18 smithi186 ceph-mon[131487]: pgmap v135: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:19.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:18 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1949311606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.479+0000 7f157e7ff700 1 -- 172.21.15.6:0/892454705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 msgr2=0x7f1578073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.479+0000 7f157e7ff700 1 --2- 172.21.15.6:0/892454705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f1578073200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1568009990 tx=0x7f1568009ca0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 -- 172.21.15.6:0/892454705 shutdown_connections 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 --2- 172.21.15.6:0/892454705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 0x7f157810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 --2- 172.21.15.6:0/892454705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f1578073200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 --2- 172.21.15.6:0/892454705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 0x7f157810c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 -- 172.21.15.6:0/892454705 >> 172.21.15.6:0/892454705 conn(0x7f15780fbd10 msgr2=0x7f15780fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 -- 172.21.15.6:0/892454705 shutdown_connections 2023-11-17T00:13:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.480+0000 7f157e7ff700 1 -- 172.21.15.6:0/892454705 wait complete. 2023-11-17T00:13:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 Processor -- start 2023-11-17T00:13:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 -- start start 2023-11-17T00:13:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 0x7f1578198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 0x7f157819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819d580 con 0x7f157810caf0 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819d6f0 con 0x7f1578072e30 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.481+0000 7f157e7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157819d8b0 con 0x7f1578073740 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44672/0 (socket says 172.21.15.6:44672) 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 -- 172.21.15.6:0/2443775670 learned_addr learned my addr 172.21.15.6:0/2443775670 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f157cd9c700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 0x7f157819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f15777fe700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 0x7f1578198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 -- 172.21.15.6:0/2443775670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 msgr2=0x7f1578198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 0x7f1578198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 -- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 msgr2=0x7f157819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 0x7f157819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 -- 172.21.15.6:0/2443775670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680095f0 con 0x7f1578072e30 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f1577fff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1568005910 tx=0x7f1568004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.482+0000 7f15757fa700 1 -- 172.21.15.6:0/2443775670 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156801d070 con 0x7f1578072e30 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.483+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578112510 con 0x7f1578072e30 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.483+0000 7f15757fa700 1 -- 172.21.15.6:0/2443775670 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1568022470 con 0x7f1578072e30 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.483+0000 7f15757fa700 1 -- 172.21.15.6:0/2443775670 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568017650 con 0x7f1578072e30 2023-11-17T00:13:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.483+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578112380 con 0x7f1578072e30 2023-11-17T00:13:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.486+0000 7f15757fa700 1 -- 172.21.15.6:0/2443775670 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f156800fad0 con 0x7f1578072e30 2023-11-17T00:13:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.488+0000 7f15920beb80 1 Processor -- start 2023-11-17T00:13:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.488+0000 7f15920beb80 1 -- start start 2023-11-17T00:13:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.488+0000 7f15757fa700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 0x7f15600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.488+0000 7f15757fa700 1 -- 172.21.15.6:0/2443775670 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f15680ce5e0 con 0x7f1578072e30 2023-11-17T00:13:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d18ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f157cd9c700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 0x7f15600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d18ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d17f8000 con 0x55c4d18fb910 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d184a840 con 0x55c4d18fb4a0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15920beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d1843cd0 con 0x55c4d174e6b0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f1577fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f1577fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:38484/0 (socket says 172.21.15.6:38484) 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f1577fff700 1 -- 172.21.15.6:0/3079548565 learned_addr learned my addr 172.21.15.6:0/3079548565 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d18ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f157cd9c700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d18ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 -- 172.21.15.6:0/3079548565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 msgr2=0x55c4d18faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 -- 172.21.15.6:0/3079548565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 msgr2=0x55c4d18ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d18ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f15777fe700 1 -- 172.21.15.6:0/3079548565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4d16c55d0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f1577fff700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.489+0000 7f157cd9c700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d18ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.490+0000 7f157cd9c700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 0x7f15600a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f157806a250 tx=0x7f1570009920 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.490+0000 7f15777fe700 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d18ff5f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f156c00bc40 tx=0x7f156c00bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43cdc78a66989809 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.490+0000 7f155e7fc700 1 -- 172.21.15.6:0/3079548565 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00fb70 con 0x55c4d18fb4a0 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.490+0000 7f155e7fc700 1 -- 172.21.15.6:0/3079548565 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c00fcd0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.490+0000 7f155e7fc700 1 -- 172.21.15.6:0/3079548565 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0116d0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 -- 172.21.15.6:0/3079548565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 msgr2=0x55c4d18ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d18ff5f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f156c00bc40 tx=0x7f156c00bf50 comp rx=0 tx=0).stop 2023-11-17T00:13:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 -- 172.21.15.6:0/3079548565 shutdown_connections 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d18ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d174e6b0 0x55c4d18faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 --2- 172.21.15.6:0/3079548565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d18ff5f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 -- 172.21.15.6:0/3079548565 >> 172.21.15.6:0/3079548565 conn(0x55c4d18e7f50 msgr2=0x55c4d18ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 -- 172.21.15.6:0/3079548565 shutdown_connections 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.491+0000 7f15920beb80 1 -- 172.21.15.6:0/3079548565 wait complete. 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 -- start start 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 Processor -- start 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb4a0 0x55c4d1904500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d1904a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d17e5880 con 0x55c4d18fb910 2023-11-17T00:13:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d17e5520 con 0x55c4d174e6b0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d16c78c0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.492+0000 7f15920beb80 10 client.0 _ll_register_callbacks cb 0x55c4d1803720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f15777fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb4a0 0x55c4d1904500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44696/0 (socket says 172.21.15.6:44696) 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 -- 172.21.15.6:0/125527271 learned_addr learned my addr 172.21.15.6:0/125527271 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f157cd9c700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d1904a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 -- 172.21.15.6:0/125527271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb4a0 msgr2=0x55c4d1904500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb4a0 0x55c4d1904500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 -- 172.21.15.6:0/125527271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 msgr2=0x55c4d1904a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d1904a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 -- 172.21.15.6:0/125527271 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c00b8a0 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f1577fff700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1568005f50 tx=0x7f156809af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf48494fe8a0534 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f15577fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156809b390 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f156809f3c0 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.493+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4d18e5710 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15680a3750 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 10 client.26378 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c4d180a130 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4d18e5710 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f156809b5f0 con 0x55c4d174e6b0 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 10 client.26378 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 20 client.26378 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1914390 0x55c4d19167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 4 client.26378 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4d1806040 con 0x55c4d1914390 2023-11-17T00:13:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.494+0000 7f15777fe700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1914390 0x55c4d19167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.495+0000 7f15777fe700 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1914390 0x55c4d19167a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d1c28e13323022 server_cookie=81f2ccd4cf0243c3 in_seq=0 out_seq=0 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.495+0000 7f15577fe700 10 client.26378 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.497+0000 7f15577fe700 1 -- 172.21.15.6:0/125527271 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c4d1806040 con 0x55c4d1914390 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.497+0000 7f15577fe700 10 client.26378 handle_command_reply: tid=0 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.497+0000 7f15920beb80 1 client.26378 shutdown 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 msgr2=0x55c4d19071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1568005f50 tx=0x7f156809af50 comp rx=0 tx=0).stop 2023-11-17T00:13:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 shutdown_connections 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1914390 0x55c4d19167a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d18fb910 0x55c4d1904a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d174e6b0 0x55c4d19071e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15577fe700 0 client.26378 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 --2- 172.21.15.6:0/125527271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb4a0 0x55c4d1904500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 >> 172.21.15.6:0/125527271 conn(0x55c4d18e7f50 msgr2=0x55c4d1900fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 shutdown_connections 2023-11-17T00:13:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 1 -- 172.21.15.6:0/125527271 wait complete. 2023-11-17T00:13:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.498+0000 7f15920beb80 20 client.26378 trim_cache size 0 max 16384 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 Processor -- start 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 -- start start 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d19044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d18ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d18e5250 con 0x55c4d174e6b0 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d1902a80 con 0x55c4d18fb4a0 2023-11-17T00:13:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15920beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d18fed20 con 0x55c4d18fb910 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15777fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f1577fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d19044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15777fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44704/0 (socket says 172.21.15.6:44704) 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f15777fe700 1 -- 172.21.15.6:0/1105065715 learned_addr learned my addr 172.21.15.6:0/1105065715 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.514+0000 7f157cd9c700 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d18ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 -- 172.21.15.6:0/1105065715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 msgr2=0x55c4d18ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d18ecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 -- 172.21.15.6:0/1105065715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 msgr2=0x55c4d19044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d19044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 -- 172.21.15.6:0/1105065715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4d18e96f0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f157cd9c700 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d18ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f15777fe700 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f156c000c00 tx=0x7f156c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeaa6bdd32790bd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f155effd700 1 -- 172.21.15.6:0/1105065715 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00faf0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f155effd700 1 -- 172.21.15.6:0/1105065715 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c027070 con 0x55c4d18fb4a0 2023-11-17T00:13:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.515+0000 7f155effd700 1 -- 172.21.15.6:0/1105065715 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c005ee0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 -- 172.21.15.6:0/1105065715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 msgr2=0x55c4d190b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f156c000c00 tx=0x7f156c0058e0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 -- 172.21.15.6:0/1105065715 shutdown_connections 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d19044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d18ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 --2- 172.21.15.6:0/1105065715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d190b950 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 -- 172.21.15.6:0/1105065715 >> 172.21.15.6:0/1105065715 conn(0x55c4d18e7f50 msgr2=0x55c4d1903da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 -- 172.21.15.6:0/1105065715 shutdown_connections 2023-11-17T00:13:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.516+0000 7f15920beb80 1 -- 172.21.15.6:0/1105065715 wait complete. 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 -- start start 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 Processor -- start 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d1909940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d1909e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d190bd00 con 0x55c4d174e6b0 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d190a3c0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d190a580 con 0x55c4d18fb910 2023-11-17T00:13:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.517+0000 7f15920beb80 10 client.0 _ll_register_callbacks cb 0x55c4d1803720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f1577fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d1909940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f1577fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:48184/0 (socket says 172.21.15.6:48184) 2023-11-17T00:13:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f1577fff700 1 -- 172.21.15.6:0/3879194226 learned_addr learned my addr 172.21.15.6:0/3879194226 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f157cd9c700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d1909e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 -- 172.21.15.6:0/3879194226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 msgr2=0x55c4d1909e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d1909e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 -- 172.21.15.6:0/3879194226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 msgr2=0x55c4d190b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 -- 172.21.15.6:0/3879194226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4d16c55d0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f1577fff700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15777fe700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d1909940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55c4d1904a10 tx=0x7f156c01aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5694397760422273 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.518+0000 7f15577fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00b500 con 0x55c4d18fb4a0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f156c0271e0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f156c00fcd0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156c0273a0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 10 client.26384 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c4d1671d80 con 0x55c4d18fb4a0 2023-11-17T00:13:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c00fcd0 con 0x55c4d18fb4a0 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f156c02e070 con 0x55c4d18fb4a0 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 10 client.26384 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 20 client.26384 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1969080 0x55c4d196b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 4 client.26384 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:13:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.519+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4d1812600 con 0x55c4d1969080 2023-11-17T00:13:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.520+0000 7f1577fff700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1969080 0x55c4d196b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.520+0000 7f1577fff700 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1969080 0x55c4d196b460 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e2c79e4bb6328e server_cookie=99095551835dc694 in_seq=0 out_seq=0 2023-11-17T00:13:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:19.520+0000 7f15577fe700 10 client.26384 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 -- 172.21.15.6:0/3164072810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c074b00 msgr2=0x7f015c073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 --2- 172.21.15.6:0/3164072810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c074b00 0x7f015c073260 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f014400b0a0 tx=0x7f014400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 -- 172.21.15.6:0/3164072810 shutdown_connections 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 --2- 172.21.15.6:0/3164072810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c10ce50 0x7f015c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 --2- 172.21.15.6:0/3164072810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c0746a0 0x7f015c072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 --2- 172.21.15.6:0/3164072810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c074b00 0x7f015c073260 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 -- 172.21.15.6:0/3164072810 >> 172.21.15.6:0/3164072810 conn(0x7f015c077500 msgr2=0x7f015c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 -- 172.21.15.6:0/3164072810 shutdown_connections 2023-11-17T00:13:19.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.569+0000 7f01613ff700 1 -- 172.21.15.6:0/3164072810 wait complete. 2023-11-17T00:13:19.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.570+0000 7f01613ff700 1 Processor -- start 2023-11-17T00:13:19.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.570+0000 7f01613ff700 1 -- start start 2023-11-17T00:13:19.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c0746a0 0x7f015c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c10ce50 0x7f015c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c198e20 con 0x7f015c0746a0 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c198f90 con 0x7f015c074b00 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01613ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c199150 con 0x7f015c10ce50 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f015affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c0746a0 0x7f015c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44714/0 (socket says 172.21.15.6:44714) 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f01527fc700 1 -- 172.21.15.6:0/3212459596 learned_addr learned my addr 172.21.15.6:0/3212459596 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.571+0000 7f015b7fe700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c10ce50 0x7f015c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 -- 172.21.15.6:0/3212459596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c10ce50 msgr2=0x7f015c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c10ce50 0x7f015c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 -- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c0746a0 msgr2=0x7f015c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c0746a0 0x7f015c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 -- 172.21.15.6:0/3212459596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0144009d00 con 0x7f015c074b00 2023-11-17T00:13:19.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01527fc700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f014400b070 tx=0x7f014400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f0158ff9700 1 -- 172.21.15.6:0/3212459596 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014400e040 con 0x7f015c074b00 2023-11-17T00:13:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c112530 con 0x7f015c074b00 2023-11-17T00:13:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f0158ff9700 1 -- 172.21.15.6:0/3212459596 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0144007930 con 0x7f015c074b00 2023-11-17T00:13:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f0158ff9700 1 -- 172.21.15.6:0/3212459596 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014401b6c0 con 0x7f015c074b00 2023-11-17T00:13:19.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.572+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1123a0 con 0x7f015c074b00 2023-11-17T00:13:19.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.576+0000 7f0158ff9700 1 -- 172.21.15.6:0/3212459596 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f014401d470 con 0x7f015c074b00 2023-11-17T00:13:19.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.577+0000 7f0158ff9700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 0x7f01480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.577+0000 7f0174dc2b80 1 Processor -- start 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.577+0000 7f0158ff9700 1 -- 172.21.15.6:0/3212459596 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f01440ce1d0 con 0x7f015c074b00 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.577+0000 7f0174dc2b80 1 -- start start 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.577+0000 7f015affd700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 0x7f01480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fc0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fc5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155ec60e0 con 0x55d155fc5930 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155f18920 con 0x55d155fc14f0 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155f11db0 con 0x55d155fc0be0 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44726/0 (socket says 172.21.15.6:44726) 2023-11-17T00:13:19.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 -- 172.21.15.6:0/166502981 learned_addr learned my addr 172.21.15.6:0/166502981 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f015affd700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 0x7f01480a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f014c007be0 tx=0x7f014c0073d0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f015b7fe700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fc5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f015affd700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fc0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 -- 172.21.15.6:0/166502981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 msgr2=0x55d155fc0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fc0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 -- 172.21.15.6:0/166502981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 msgr2=0x55d155fc5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fc5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f01527fc700 1 -- 172.21.15.6:0/166502981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d155d936b0 con 0x55d155fc14f0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.578+0000 7f015b7fe700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fc5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f01527fc700 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01440979f0 tx=0x7f0144003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3cbb2a9158b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f01517fa700 1 -- 172.21.15.6:0/166502981 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01440a3340 con 0x55d155fc14f0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f01517fa700 1 -- 172.21.15.6:0/166502981 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0144097690 con 0x55d155fc14f0 2023-11-17T00:13:19.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f01517fa700 1 -- 172.21.15.6:0/166502981 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014409a720 con 0x55d155fc14f0 2023-11-17T00:13:19.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f0174dc2b80 1 -- 172.21.15.6:0/166502981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 msgr2=0x55d155fc53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.579+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01440979f0 tx=0x7f0144003e10 comp rx=0 tx=0).stop 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 -- 172.21.15.6:0/166502981 shutdown_connections 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fc5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fc0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/166502981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc53f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 -- 172.21.15.6:0/166502981 >> 172.21.15.6:0/166502981 conn(0x55d155fb63e0 msgr2=0x55d155fbbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 -- 172.21.15.6:0/166502981 shutdown_connections 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 -- 172.21.15.6:0/166502981 wait complete. 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 -- start start 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.580+0000 7f0174dc2b80 1 Processor -- start 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155eb3960 con 0x55d155fc5930 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155eb3600 con 0x55d155fc14f0 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155d959a0 con 0x55d155fc0be0 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f0174dc2b80 10 client.0 _ll_register_callbacks cb 0x55d155ed1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f015b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f015affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44738/0 (socket says 172.21.15.6:44738) 2023-11-17T00:13:19.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f01527fc700 1 -- 172.21.15.6:0/3923263400 learned_addr learned my addr 172.21.15.6:0/3923263400 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:19.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f01527fc700 1 -- 172.21.15.6:0/3923263400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 msgr2=0x55d155fca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.581+0000 7f01527fc700 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01527fc700 1 -- 172.21.15.6:0/3923263400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 msgr2=0x55d155fcae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01527fc700 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fcae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01527fc700 1 -- 172.21.15.6:0/3923263400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1123a0 con 0x55d155fc14f0 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01527fc700 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0144006070 tx=0x7f014401db80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea15e236dd194b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01427fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:19.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01440a3340 con 0x55d155fc14f0 2023-11-17T00:13:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d155fb3ba0 con 0x55d155fc14f0 2023-11-17T00:13:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0144099320 con 0x55d155fc14f0 2023-11-17T00:13:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01440a30c0 con 0x55d155fc14f0 2023-11-17T00:13:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f0174dc2b80 10 client.26396 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:19.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d155ed8210 con 0x55d155fc14f0 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.582+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01440cb020 con 0x55d155fc14f0 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f014401d720 con 0x55d155fc14f0 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 10 client.26396 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 20 client.26396 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d155fda290 0x55d155fdc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:19.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 4 client.26396 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:19.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d155ed4120 con 0x55d155fda290 2023-11-17T00:13:19.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.583+0000 7f015affd700 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d155fda290 0x55d155fdc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:19.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.584+0000 7f015affd700 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d155fda290 0x55d155fdc6a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d88e18514f8e10a server_cookie=d1a40b070b132c8d in_seq=0 out_seq=0 2023-11-17T00:13:19.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:19.584+0000 7f01427fc700 10 client.26396 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:20 smithi006 ceph-mon[97840]: pgmap v136: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:21.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:20 smithi087 ceph-mon[131182]: pgmap v136: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:21.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:20 smithi186 ceph-mon[131487]: pgmap v136: 129 pgs: 129 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2023-11-17T00:13:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:22 smithi006 ceph-mon[97840]: pgmap v137: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2023-11-17T00:13:23.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:22 smithi087 ceph-mon[131182]: pgmap v137: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2023-11-17T00:13:23.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:22 smithi186 ceph-mon[131487]: pgmap v137: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2023-11-17T00:13:25.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:24 smithi006 ceph-mon[97840]: pgmap v138: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 20 op/s 2023-11-17T00:13:25.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:24 smithi087 ceph-mon[131182]: pgmap v138: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 20 op/s 2023-11-17T00:13:25.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:24 smithi186 ceph-mon[131487]: pgmap v138: 129 pgs: 129 active+clean; 55 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 20 op/s 2023-11-17T00:13:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:26 smithi006 ceph-mon[97840]: pgmap v139: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 1.26k op/s 2023-11-17T00:13:27.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:26 smithi087 ceph-mon[131182]: pgmap v139: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 1.26k op/s 2023-11-17T00:13:27.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:26 smithi186 ceph-mon[131487]: pgmap v139: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 1.26k op/s 2023-11-17T00:13:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.747+0000 7f15577fe700 1 -- 172.21.15.6:0/3879194226 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4d1812600 con 0x55c4d1969080 2023-11-17T00:13:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.747+0000 7f15577fe700 10 client.26384 handle_command_reply: tid=0 2023-11-17T00:13:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.747+0000 7f15920beb80 1 client.26384 shutdown 2023-11-17T00:13:28.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f01427fc700 1 -- 172.21.15.6:0/3923263400 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d155ed4120 con 0x55d155fda290 2023-11-17T00:13:28.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f01427fc700 10 client.26396 handle_command_reply: tid=0 2023-11-17T00:13:28.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f0174dc2b80 1 client.26396 shutdown 2023-11-17T00:13:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 msgr2=0x55c4d1909940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d1909940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55c4d1904a10 tx=0x7f156c01aec0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 shutdown_connections 2023-11-17T00:13:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c4d1969080 0x55c4d196b460 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c4d174e6b0 0x55c4d190b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c4d18fb910 0x55c4d1909e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 --2- 172.21.15.6:0/3879194226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c4d18fb4a0 0x55c4d1909940 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55c4d1904a10 tx=0x7f156c01aec0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 >> 172.21.15.6:0/3879194226 conn(0x55c4d18e7f50 msgr2=0x55c4d1902cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f15577fe700 0 client.26384 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:28.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 msgr2=0x55d155fca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:28.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.748+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0144006070 tx=0x7f014401db80 comp rx=0 tx=0).stop 2023-11-17T00:13:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 shutdown_connections 2023-11-17T00:13:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f15920beb80 1 -- 172.21.15.6:0/3879194226 wait complete. 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 shutdown_connections 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d155fda290 0x55d155fdc6a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fcae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/3923263400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fca8e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 >> 172.21.15.6:0/3923263400 conn(0x55d155fb63e0 msgr2=0x55d155fc6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:28.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.749+0000 7f01427fc700 0 client.26396 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.750+0000 7f15920beb80 20 client.26384 trim_cache size 0 max 16384 2023-11-17T00:13:28.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.750+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 shutdown_connections 2023-11-17T00:13:28.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.750+0000 7f0174dc2b80 1 -- 172.21.15.6:0/3923263400 wait complete. 2023-11-17T00:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:13:28.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.750+0000 7f0174dc2b80 20 client.26396 trim_cache size 0 max 16384 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 msgr2=0x7f15600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 0x7f15600a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f157806a250 tx=0x7f1570009920 comp rx=0 tx=0).stop 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 msgr2=0x7f15781985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1568005910 tx=0x7f1568004730 comp rx=0 tx=0).stop 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 shutdown_connections 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f15600a12c0 0x7f15600a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f157810caf0 0x7f157819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1578072e30 0x7f15781985a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 --2- 172.21.15.6:0/2443775670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578073740 0x7f1578198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 >> 172.21.15.6:0/2443775670 conn(0x7f15780fbd10 msgr2=0x7f1578104550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 shutdown_connections 2023-11-17T00:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:28.751+0000 7f157e7ff700 1 -- 172.21.15.6:0/2443775670 wait complete. 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.771+0000 7f0174dc2b80 1 Processor -- start 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.771+0000 7f0174dc2b80 1 -- start start 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fd1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155fc1c40 con 0x55d155fc5930 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155fc8900 con 0x55d155fc14f0 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d155fc44c0 con 0x55d155fc0be0 2023-11-17T00:13:28.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f01527fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44742/0 (socket says 172.21.15.6:44742) 2023-11-17T00:13:28.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f01527fc700 1 -- 172.21.15.6:0/1417280174 learned_addr learned my addr 172.21.15.6:0/1417280174 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:28.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f015affd700 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.772+0000 7f015b7fe700 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fd1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 -- 172.21.15.6:0/1417280174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 msgr2=0x55d155fbbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:28.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fbbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 -- 172.21.15.6:0/1417280174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 msgr2=0x55d155fd1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:28.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fd1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 -- 172.21.15.6:0/1417280174 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d155fb7b80 con 0x55d155fc14f0 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f01527fc700 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0144097480 tx=0x7f01440ba000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a26c96de0499e9f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f0151ffb700 1 -- 172.21.15.6:0/1417280174 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144099320 con 0x55d155fc14f0 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f0151ffb700 1 -- 172.21.15.6:0/1417280174 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01440cb040 con 0x55d155fc14f0 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.773+0000 7f0151ffb700 1 -- 172.21.15.6:0/1417280174 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01440a3340 con 0x55d155fc14f0 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 -- 172.21.15.6:0/1417280174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 msgr2=0x55d155fc7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0144097480 tx=0x7f01440ba000 comp rx=0 tx=0).stop 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 -- 172.21.15.6:0/1417280174 shutdown_connections 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc5930 0x55d155fd1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc0be0 0x55d155fbbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/1417280174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc14f0 0x55d155fc7e80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 -- 172.21.15.6:0/1417280174 >> 172.21.15.6:0/1417280174 conn(0x55d155fb63e0 msgr2=0x55d155fc48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:28.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 -- 172.21.15.6:0/1417280174 shutdown_connections 2023-11-17T00:13:28.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.774+0000 7f0174dc2b80 1 -- 172.21.15.6:0/1417280174 wait complete. 2023-11-17T00:13:28.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.775+0000 7f0174dc2b80 1 -- start start 2023-11-17T00:13:28.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.775+0000 7f0174dc2b80 1 Processor -- start 2023-11-17T00:13:28.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 0x55d155fcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc14f0 0x55d155fce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15602e1e0 con 0x55d155fc0be0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15602e350 con 0x55d155fc5930 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15602e510 con 0x55d155fc14f0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f0174dc2b80 10 client.0 _ll_register_callbacks cb 0x55d155ed1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44746/0 (socket says 172.21.15.6:44746) 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 -- 172.21.15.6:0/438611102 learned_addr learned my addr 172.21.15.6:0/438611102 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f01527fc700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc14f0 0x55d155fce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015affd700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 0x55d155fcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 -- 172.21.15.6:0/438611102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc14f0 msgr2=0x55d155fce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc14f0 0x55d155fce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 -- 172.21.15.6:0/438611102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 msgr2=0x55d155fcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 0x55d155fcdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:28.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015b7fe700 1 -- 172.21.15.6:0/438611102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d155ed8210 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.776+0000 7f015affd700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 0x55d155fcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f015b7fe700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01540042c0 tx=0x7f01540045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7638668c88c94b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f01427fc700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154020070 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0144097690 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0154004830 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0154016040 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f0174dc2b80 10 client.26402 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.777+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d155d3fe60 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0144097690 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0154023020 con 0x55d155fc5930 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 10 client.26402 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:28.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 20 client.26402 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d15602fca0 0x55d1560320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 4 client.26402 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d155ee06e0 con 0x55d15602fca0 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.778+0000 7f015affd700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d15602fca0 0x55d1560320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.779+0000 7f015affd700 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d15602fca0 0x55d1560320b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a849a642584214 server_cookie=af754910bc4d516f in_seq=0 out_seq=0 2023-11-17T00:13:28.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:28.779+0000 7f01427fc700 10 client.26402 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:28 smithi006 ceph-mon[97840]: pgmap v140: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:29.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:28 smithi087 ceph-mon[131182]: pgmap v140: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:29.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:28 smithi186 ceph-mon[131487]: pgmap v140: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:30.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f01427fc700 1 -- 172.21.15.6:0/438611102 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d155ee06e0 con 0x55d15602fca0 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f01427fc700 10 client.26402 handle_command_reply: tid=0 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 client.26402 shutdown 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 msgr2=0x55d155fcea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01540042c0 tx=0x7f01540045d0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 shutdown_connections 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d15602fca0 0x55d1560320b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d155fc0be0 0x55d155fcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f01427fc700 0 client.26402 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d155fc5930 0x55d155fcea70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 --2- 172.21.15.6:0/438611102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d155fc14f0 0x55d155fce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 >> 172.21.15.6:0/438611102 conn(0x55d155fb63e0 msgr2=0x55d155fc7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 shutdown_connections 2023-11-17T00:13:30.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.729+0000 7f0174dc2b80 1 -- 172.21.15.6:0/438611102 wait complete. 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f0174dc2b80 20 client.26402 trim_cache size 0 max 16384 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:13:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 msgr2=0x7f01480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 0x7f01480a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f014c007be0 tx=0x7f014c0073d0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 msgr2=0x7f015c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f014400b070 tx=0x7f014400bce0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 shutdown_connections 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f01480a12c0 0x7f01480a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c0746a0 0x7f015c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f015c10ce50 0x7f015c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 --2- 172.21.15.6:0/3212459596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f015c074b00 0x7f015c19ebb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.730+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 >> 172.21.15.6:0/3212459596 conn(0x7f015c077500 msgr2=0x7f015c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:30.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.731+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 shutdown_connections 2023-11-17T00:13:30.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.731+0000 7f01613ff700 1 -- 172.21.15.6:0/3212459596 wait complete. 2023-11-17T00:13:30.751 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:13:30.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 -- 172.21.15.6:0/2564085604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 msgr2=0x7f6bc4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/2564085604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4104e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80099c0 tx=0x7f6bb8009cd0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 -- 172.21.15.6:0/2564085604 shutdown_connections 2023-11-17T00:13:30.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/2564085604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4105560 0x7f6bc410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/2564085604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4100c50 0x7f6bc4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/2564085604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4104e80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 -- 172.21.15.6:0/2564085604 >> 172.21.15.6:0/2564085604 conn(0x7f6bc40774f0 msgr2=0x7f6bc40778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 -- 172.21.15.6:0/2564085604 shutdown_connections 2023-11-17T00:13:30.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.973+0000 7f6bca7ff700 1 -- 172.21.15.6:0/2564085604 wait complete. 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.974+0000 7f6bca7ff700 1 Processor -- start 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.974+0000 7f6bca7ff700 1 -- start start 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4100c50 0x7f6bc41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4105560 0x7f6bc419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419d5b0 con 0x7f6bc4100c50 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419d720 con 0x7f6bc41015f0 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bca7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419d8e0 con 0x7f6bc4105560 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4100c50 0x7f6bc41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46810/0 (socket says 172.21.15.6:46810) 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 -- 172.21.15.6:0/885696232 learned_addr learned my addr 172.21.15.6:0/885696232 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4105560 0x7f6bc419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 -- 172.21.15.6:0/885696232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4105560 msgr2=0x7f6bc419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4105560 0x7f6bc419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 -- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4100c50 msgr2=0x7f6bc41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4100c50 0x7f6bc41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.975+0000 7f6bc37fe700 1 -- 172.21.15.6:0/885696232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb80095f0 con 0x7f6bc41015f0 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098e0 tx=0x7f6bb800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:30.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bc17fa700 1 -- 172.21.15.6:0/885696232 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb801d070 con 0x7f6bc41015f0 2023-11-17T00:13:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bc17fa700 1 -- 172.21.15.6:0/885696232 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb800fb80 con 0x7f6bc41015f0 2023-11-17T00:13:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bc17fa700 1 -- 172.21.15.6:0/885696232 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8022ea0 con 0x7f6bc41015f0 2023-11-17T00:13:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc4110340 con 0x7f6bc41015f0 2023-11-17T00:13:30.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.976+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc41101b0 con 0x7f6bc41015f0 2023-11-17T00:13:30.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.979+0000 7f6bc17fa700 1 -- 172.21.15.6:0/885696232 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6bb800fcf0 con 0x7f6bc41015f0 2023-11-17T00:13:30.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.980+0000 7f6bc17fa700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 0x7f6bac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 Processor -- start 2023-11-17T00:13:30.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bc17fa700 1 -- 172.21.15.6:0/885696232 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6bb80cdff0 con 0x7f6bc41015f0 2023-11-17T00:13:30.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 0x7f6bac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 -- start start 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c232f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c237d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c138060 con 0x56196c2378b0 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c18a8a0 con 0x56196c233470 2023-11-17T00:13:30.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c183d30 con 0x56196c232b60 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.981+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 0x7f6bac0a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009c60 tx=0x7f6bb4009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46816/0 (socket says 172.21.15.6:46816) 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3276645153 learned_addr learned my addr 172.21.15.6:0/3276645153 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c237d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c232f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3276645153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 msgr2=0x56196c232f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c232f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3276645153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 msgr2=0x56196c237d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c237d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3276645153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196c005630 con 0x56196c233470 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c237d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:30.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8097810 tx=0x7f6bb809aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4755e21d11898327 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6baa7fc700 1 -- 172.21.15.6:0/3276645153 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80a3160 con 0x56196c233470 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6baa7fc700 1 -- 172.21.15.6:0/3276645153 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb809b000 con 0x56196c233470 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.982+0000 7f6baa7fc700 1 -- 172.21.15.6:0/3276645153 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb809a560 con 0x56196c233470 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3276645153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 msgr2=0x56196c237370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8097810 tx=0x7f6bb809aa10 comp rx=0 tx=0).stop 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3276645153 shutdown_connections 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c237d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c232f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3276645153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c237370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3276645153 >> 172.21.15.6:0/3276645153 conn(0x56196c228360 msgr2=0x56196c22db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3276645153 shutdown_connections 2023-11-17T00:13:30.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.983+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3276645153 wait complete. 2023-11-17T00:13:30.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.984+0000 7f6bde16eb80 1 -- start start 2023-11-17T00:13:30.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.984+0000 7f6bde16eb80 1 Processor -- start 2023-11-17T00:13:30.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.984+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c23ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c23c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c1258e0 con 0x56196c2378b0 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c125580 con 0x56196c233470 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c007920 con 0x56196c232b60 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bde16eb80 10 client.0 _ll_register_callbacks cb 0x56196c143780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc8d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c23c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46830/0 (socket says 172.21.15.6:46830) 2023-11-17T00:13:30.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 -- 172.21.15.6:0/2614086998 learned_addr learned my addr 172.21.15.6:0/2614086998 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c23ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 -- 172.21.15.6:0/2614086998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 msgr2=0x56196c23ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c23ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 -- 172.21.15.6:0/2614086998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 msgr2=0x56196c23c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c23c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 -- 172.21.15.6:0/2614086998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc41101b0 con 0x56196c233470 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c23ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80a2c30 tx=0x7f6bb80cb370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dffa9a27fb590e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.985+0000 7f6ba37fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:30.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80a3160 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6bb80cbae0 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56196c225b20 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb80cbca0 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 10 client.26414 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56196c14a190 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb80cb020 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bb8022470 con 0x56196c233470 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 10 client.26414 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:30.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 20 client.26414 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c24c150 0x56196c24e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 4 client.26414 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.986+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56196c1460a0 con 0x56196c24c150 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.987+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c24c150 0x56196c24e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.987+0000 7f6ba37fe700 10 client.26414 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:30.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.987+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c24c150 0x56196c24e530 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6262a4fd6af2d54b server_cookie=fd33b926a8b65022 in_seq=0 out_seq=0 2023-11-17T00:13:30.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6ba37fe700 1 -- 172.21.15.6:0/2614086998 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56196c1460a0 con 0x56196c24c150 2023-11-17T00:13:30.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6ba37fe700 10 client.26414 handle_command_reply: tid=0 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 client.26414 shutdown 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 msgr2=0x56196c23c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80a2c30 tx=0x7f6bb80cb370 comp rx=0 tx=0).stop 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 shutdown_connections 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c24c150 0x56196c24e530 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c23c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c23ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6ba37fe700 0 client.26414 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/2614086998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c23c1f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.990+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 >> 172.21.15.6:0/2614086998 conn(0x56196c228360 msgr2=0x56196c238d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.991+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 shutdown_connections 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.991+0000 7f6bde16eb80 1 -- 172.21.15.6:0/2614086998 wait complete. 2023-11-17T00:13:31.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:30.991+0000 7f6bde16eb80 20 client.26414 trim_cache size 0 max 16384 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 Processor -- start 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 -- start start 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c233470 0x56196c22da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 0x56196c232070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c22f370 con 0x56196c232b60 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c233880 con 0x56196c2378b0 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c227ee0 con 0x56196c233470 2023-11-17T00:13:31.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bc8d9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 0x56196c232070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45338/0 (socket says 172.21.15.6:45338) 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bc3fff700 1 -- 172.21.15.6:0/4050849683 learned_addr learned my addr 172.21.15.6:0/4050849683 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.009+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c233470 0x56196c22da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 -- 172.21.15.6:0/4050849683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c233470 msgr2=0x56196c22da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c233470 0x56196c22da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 -- 172.21.15.6:0/4050849683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 msgr2=0x56196c2436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 -- 172.21.15.6:0/4050849683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196c149ee0 con 0x56196c2378b0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 0x56196c232070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0094e0 tx=0x7f6bbc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd7d6314901ab3c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6baaffd700 1 -- 172.21.15.6:0/4050849683 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc01b070 con 0x56196c2378b0 2023-11-17T00:13:31.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6baaffd700 1 -- 172.21.15.6:0/4050849683 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bbc00f040 con 0x56196c2378b0 2023-11-17T00:13:31.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.010+0000 7f6baaffd700 1 -- 172.21.15.6:0/4050849683 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00e040 con 0x56196c2378b0 2023-11-17T00:13:31.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- 172.21.15.6:0/4050849683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 msgr2=0x56196c232070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 0x56196c232070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0094e0 tx=0x7f6bbc00b040 comp rx=0 tx=0).stop 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- 172.21.15.6:0/4050849683 shutdown_connections 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c232b60 0x56196c2436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c2378b0 0x56196c232070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/4050849683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c233470 0x56196c22da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- 172.21.15.6:0/4050849683 >> 172.21.15.6:0/4050849683 conn(0x56196c228360 msgr2=0x56196c231960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- 172.21.15.6:0/4050849683 shutdown_connections 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- 172.21.15.6:0/4050849683 wait complete. 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 -- start start 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.011+0000 7f6bde16eb80 1 Processor -- start 2023-11-17T00:13:31.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c2435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c241e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c242430 con 0x56196c2378b0 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c242570 con 0x56196c233470 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c242730 con 0x56196c232b60 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bde16eb80 10 client.0 _ll_register_callbacks cb 0x7f6bb40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bc8d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c241e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46852/0 (socket says 172.21.15.6:46852) 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c2435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.012+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3479024978 learned_addr learned my addr 172.21.15.6:0/3479024978 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3479024978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 msgr2=0x56196c2435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:31.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c2435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3479024978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 msgr2=0x56196c241e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c241e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 -- 172.21.15.6:0/3479024978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb4017070 con 0x56196c233470 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc8d9c700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c241e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c2435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c010 tx=0x7f6bb8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6628170f0b65085e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6ba37fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80ba3b0 con 0x56196c233470 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6bb80cb1b0 con 0x56196c233470 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bb809b000 con 0x56196c233470 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb80cde20 con 0x56196c233470 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bde16eb80 10 client.26420 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:31.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.013+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f6bbc0091c0 con 0x56196c233470 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb809b000 con 0x56196c233470 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bb8099d00 con 0x56196c233470 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 10 client.26420 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 20 client.26420 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c2a1080 0x56196c2a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 4 client.26420 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56196c152660 con 0x56196c2a1080 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c2a1080 0x56196c2a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6ba37fe700 10 client.26420 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:31.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.014+0000 7f6bc3fff700 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c2a1080 0x56196c2a3490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1277c9d322e7a496 server_cookie=2f637d779bf71ef3 in_seq=0 out_seq=0 2023-11-17T00:13:31.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6ba37fe700 1 -- 172.21.15.6:0/3479024978 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56196c152660 con 0x56196c2a1080 2023-11-17T00:13:31.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6ba37fe700 10 client.26420 handle_command_reply: tid=0 2023-11-17T00:13:31.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 client.26420 shutdown 2023-11-17T00:13:31.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 msgr2=0x56196c241920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c010 tx=0x7f6bb8005e80 comp rx=0 tx=0).stop 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 shutdown_connections 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56196c2a1080 0x56196c2a3490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56196c2378b0 0x56196c241e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56196c232b60 0x56196c2435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 --2- 172.21.15.6:0/3479024978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56196c233470 0x56196c241920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6ba37fe700 0 client.26420 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 >> 172.21.15.6:0/3479024978 conn(0x56196c228360 msgr2=0x56196c2321f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:31.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 shutdown_connections 2023-11-17T00:13:31.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.017+0000 7f6bde16eb80 1 -- 172.21.15.6:0/3479024978 wait complete. 2023-11-17T00:13:31.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bde16eb80 20 client.26420 trim_cache size 0 max 16384 2023-11-17T00:13:31.023 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:13:31.023 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (3 inodes in the stack)", 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: "9ebf54b4-47aa-46ec-acbc-7e7519866a60": { 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:13:31.024 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:13:31.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 msgr2=0x7f6bac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:31.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 0x7f6bac0a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009c60 tx=0x7f6bb4009260 comp rx=0 tx=0).stop 2023-11-17T00:13:31.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 msgr2=0x7f6bc4198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:31.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098e0 tx=0x7f6bb800f690 comp rx=0 tx=0).stop 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 shutdown_connections 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6bac0a1340 0x7f6bac0a37f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6bc4100c50 0x7f6bc41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bc4105560 0x7f6bc419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 --2- 172.21.15.6:0/885696232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bc41015f0 0x7f6bc4198b10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 >> 172.21.15.6:0/885696232 conn(0x7f6bc40774f0 msgr2=0x7f6bc40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:31.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 shutdown_connections 2023-11-17T00:13:31.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:13:31.018+0000 7f6bca7ff700 1 -- 172.21.15.6:0/885696232 wait complete. 2023-11-17T00:13:31.037 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:30 smithi006 ceph-mon[97840]: pgmap v141: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:31.041 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebf54b4-47aa-46ec-acbc-7e7519866a60 - {'path': '/', 'tag': '9ebf54b4-47aa-46ec-acbc-7e7519866a60', 'options': 'recursive,force'} 2023-11-17T00:13:31.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:30 smithi087 ceph-mon[131182]: pgmap v141: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:31.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:30 smithi186 ceph-mon[131487]: pgmap v141: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.24k op/s 2023-11-17T00:13:32.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:31 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:13:32.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:31 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:13:32.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:31 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:13:32.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:31 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:13:32.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:31 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:13:32.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:31 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:13:32.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:31 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:13:32.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:31 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:13:32.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:31 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:13:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:32 smithi006 ceph-mon[97840]: pgmap v142: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 1.25k op/s 2023-11-17T00:13:33.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:32 smithi087 ceph-mon[131182]: pgmap v142: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 1.25k op/s 2023-11-17T00:13:33.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:32 smithi186 ceph-mon[131487]: pgmap v142: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 1.25k op/s 2023-11-17T00:13:34.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:34 smithi087 ceph-mon[131182]: pgmap v143: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 1.24k op/s 2023-11-17T00:13:34.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:34 smithi186 ceph-mon[131487]: pgmap v143: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 1.24k op/s 2023-11-17T00:13:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:34 smithi006 ceph-mon[97840]: pgmap v143: 129 pgs: 129 active+clean; 33 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 1.24k op/s 2023-11-17T00:13:36.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:36 smithi087 ceph-mon[131182]: pgmap v144: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 1.31k op/s 2023-11-17T00:13:36.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:36 smithi186 ceph-mon[131487]: pgmap v144: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 1.31k op/s 2023-11-17T00:13:36.972 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:36 smithi006 ceph-mon[97840]: pgmap v144: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 1.31k op/s 2023-11-17T00:13:37.509 INFO:tasks.workunit.client.0.smithi006.stdout:created hierarchy, now cleaning up 2023-11-17T00:13:38.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:38 smithi087 ceph-mon[131182]: pgmap v145: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:38.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:38 smithi186 ceph-mon[131487]: pgmap v145: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:38 smithi006 ceph-mon[97840]: pgmap v145: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:40.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:40 smithi087 ceph-mon[131182]: pgmap v146: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:40.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:40 smithi186 ceph-mon[131487]: pgmap v146: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:40 smithi006 ceph-mon[97840]: pgmap v146: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 74 op/s 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 -- 172.21.15.6:0/1572553294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 msgr2=0x7f581410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 --2- 172.21.15.6:0/1572553294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f581410d0b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b0d0 tx=0x7f580c00b3e0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 -- 172.21.15.6:0/1572553294 shutdown_connections 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 --2- 172.21.15.6:0/1572553294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 --2- 172.21.15.6:0/1572553294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f581410d0b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 --2- 172.21.15.6:0/1572553294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 -- 172.21.15.6:0/1572553294 >> 172.21.15.6:0/1572553294 conn(0x7f58140fbd10 msgr2=0x7f58140fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 -- 172.21.15.6:0/1572553294 shutdown_connections 2023-11-17T00:13:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.002+0000 7f581afff700 1 -- 172.21.15.6:0/1572553294 wait complete. 2023-11-17T00:13:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.003+0000 7f581afff700 1 Processor -- start 2023-11-17T00:13:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 -- start start 2023-11-17T00:13:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f58141a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814073b60 con 0x7f58141004a0 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141a4c10 con 0x7f5814100e40 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581afff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141a4d50 con 0x7f58141054e0 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f5813fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f581959c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f58141a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54758/0 (socket says 172.21.15.6:54758) 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.004+0000 7f5818d9b700 1 -- 172.21.15.6:0/150479062 learned_addr learned my addr 172.21.15.6:0/150479062 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 -- 172.21.15.6:0/150479062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 msgr2=0x7f58141a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f58141a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 -- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 msgr2=0x7f5814072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 -- 172.21.15.6:0/150479062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009d00 con 0x7f5814100e40 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5813fff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814073330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5814101210 tx=0x7f580000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5811ffb700 1 -- 172.21.15.6:0/150479062 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800010900 con 0x7f5814100e40 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814110300 con 0x7f5814100e40 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5811ffb700 1 -- 172.21.15.6:0/150479062 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f580000c430 con 0x7f5814100e40 2023-11-17T00:13:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f5811ffb700 1 -- 172.21.15.6:0/150479062 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800018680 con 0x7f5814100e40 2023-11-17T00:13:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.005+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814110170 con 0x7f5814100e40 2023-11-17T00:13:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.009+0000 7f5811ffb700 1 -- 172.21.15.6:0/150479062 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f580000c5a0 con 0x7f5814100e40 2023-11-17T00:13:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.010+0000 7f582e9b5b80 1 Processor -- start 2023-11-17T00:13:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.010+0000 7f5811ffb700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 0x7f58040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.010+0000 7f582e9b5b80 1 -- start start 2023-11-17T00:13:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.010+0000 7f5811ffb700 1 -- 172.21.15.6:0/150479062 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5800014070 con 0x7f5814100e40 2023-11-17T00:13:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f581959c700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 0x7f58040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 0x559bff54d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54db90 0x559bff547b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff44d000 con 0x559bff3a36b0 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff49f840 con 0x559bff54db90 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff498cd0 con 0x559bff54d720 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 0x559bff54d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 0x559bff54d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54768/0 (socket says 172.21.15.6:54768) 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f5818d9b700 1 -- 172.21.15.6:0/3161548880 learned_addr learned my addr 172.21.15.6:0/3161548880 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.011+0000 7f5813fff700 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 -- 172.21.15.6:0/3161548880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54db90 msgr2=0x559bff547b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54db90 0x559bff547b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 -- 172.21.15.6:0/3161548880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 msgr2=0x559bff54d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 0x559bff54d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 -- 172.21.15.6:0/3161548880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bff31a5d0 con 0x559bff54d720 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f581959c700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 0x7f58040a37f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5814074f00 tx=0x7f580c00b040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f5813fff700 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f58000ca470 tx=0x7f5800099a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194d23eac9adfe89 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f57feffd700 1 -- 172.21.15.6:0/3161548880 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a2240 con 0x559bff54d720 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f57feffd700 1 -- 172.21.15.6:0/3161548880 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f580000c880 con 0x559bff54d720 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.012+0000 7f57feffd700 1 -- 172.21.15.6:0/3161548880 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a1db0 con 0x559bff54d720 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 -- 172.21.15.6:0/3161548880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 msgr2=0x559bff547560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f58000ca470 tx=0x7f5800099a60 comp rx=0 tx=0).stop 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 -- 172.21.15.6:0/3161548880 shutdown_connections 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff3a36b0 0x559bff54d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54db90 0x559bff547b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/3161548880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 -- 172.21.15.6:0/3161548880 >> 172.21.15.6:0/3161548880 conn(0x559bff53cf50 msgr2=0x559bff542270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 -- 172.21.15.6:0/3161548880 shutdown_connections 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.013+0000 7f582e9b5b80 1 -- 172.21.15.6:0/3161548880 wait complete. 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.014+0000 7f582e9b5b80 1 -- start start 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.014+0000 7f582e9b5b80 1 Processor -- start 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff550d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff5517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff43a880 con 0x559bff54db90 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff43a520 con 0x559bff3a36b0 2023-11-17T00:13:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff31c8c0 con 0x559bff54d720 2023-11-17T00:13:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f582e9b5b80 10 client.0 _ll_register_callbacks cb 0x559bff458720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff550d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:57814/0 (socket says 172.21.15.6:57814) 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f581959c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff5517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 -- 172.21.15.6:0/1661436395 learned_addr learned my addr 172.21.15.6:0/1661436395 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 -- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 msgr2=0x559bff550d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff550d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 -- 172.21.15.6:0/1661436395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 msgr2=0x559bff5517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff5517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5813fff700 1 -- 172.21.15.6:0/1661436395 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814110170 con 0x559bff54d720 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.015+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff550d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f5813fff700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58000962a0 tx=0x7f580000ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a385421c77fd0b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:13:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a2240 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bff53a710 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f58000a1fc0 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58000cabc0 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f582e9b5b80 10 client.26395 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559bff45f130 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58000ca020 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.016+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5800095d00 con 0x559bff54d720 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 10 client.26395 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 20 client.26395 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5693b0 0x559bff56b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 4 client.26395 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:13:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bff45b040 con 0x559bff5693b0 2023-11-17T00:13:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5693b0 0x559bff56b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5693b0 0x559bff56b7c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8d7ba17b2a24a5 server_cookie=48946f77eff3f1bc in_seq=0 out_seq=0 2023-11-17T00:13:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.017+0000 7f57f3fff700 10 client.26395 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f57f3fff700 1 -- 172.21.15.6:0/1661436395 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559bff45b040 con 0x559bff5693b0 2023-11-17T00:13:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f57f3fff700 10 client.26395 handle_command_reply: tid=0 2023-11-17T00:13:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 client.26395 shutdown 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 msgr2=0x559bff5512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58000962a0 tx=0x7f580000ca60 comp rx=0 tx=0).stop 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 shutdown_connections 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5693b0 0x559bff56b7c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff5517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff550d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1661436395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff5512b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 >> 172.21.15.6:0/1661436395 conn(0x559bff53cf50 msgr2=0x559bff5417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f57f3fff700 0 client.26395 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 shutdown_connections 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.020+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1661436395 wait complete. 2023-11-17T00:13:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.021+0000 7f582e9b5b80 20 client.26395 trim_cache size 0 max 16384 2023-11-17T00:13:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 Processor -- start 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 -- start start 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff541f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff53efd0 con 0x559bff54db90 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff548220 con 0x559bff3a36b0 2023-11-17T00:13:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff544060 con 0x559bff54d720 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f5813fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:56552/0 (socket says 172.21.15.6:56552) 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f5818d9b700 1 -- 172.21.15.6:0/2183770892 learned_addr learned my addr 172.21.15.6:0/2183770892 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.037+0000 7f581959c700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff541f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 -- 172.21.15.6:0/2183770892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 msgr2=0x559bff547560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 -- 172.21.15.6:0/2183770892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 msgr2=0x559bff541f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff541f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 -- 172.21.15.6:0/2183770892 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bff53e6f0 con 0x559bff3a36b0 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f581959c700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff541f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5813fff700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f5818d9b700 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5808004790 tx=0x7f5808009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7662c7a344b82171 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f57ff7fe700 1 -- 172.21.15.6:0/2183770892 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808017040 con 0x559bff3a36b0 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f57ff7fe700 1 -- 172.21.15.6:0/2183770892 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5808018040 con 0x559bff3a36b0 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.038+0000 7f57ff7fe700 1 -- 172.21.15.6:0/2183770892 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808010a90 con 0x559bff3a36b0 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 -- 172.21.15.6:0/2183770892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 msgr2=0x559bff5608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5808004790 tx=0x7f5808009260 comp rx=0 tx=0).stop 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 -- 172.21.15.6:0/2183770892 shutdown_connections 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff541f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff3a36b0 0x559bff5608a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/2183770892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff54d720 0x559bff547560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 -- 172.21.15.6:0/2183770892 >> 172.21.15.6:0/2183770892 conn(0x559bff53cf50 msgr2=0x559bff56b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 -- 172.21.15.6:0/2183770892 shutdown_connections 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.039+0000 7f582e9b5b80 1 -- 172.21.15.6:0/2183770892 wait complete. 2023-11-17T00:13:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 -- start start 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 Processor -- start 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 0x559bff560aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff549140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff549810 con 0x559bff54db90 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff549980 con 0x559bff54d720 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bff549b40 con 0x559bff3a36b0 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f582e9b5b80 10 client.0 _ll_register_callbacks cb 0x7f58080098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f5813fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f5818d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 0x559bff560aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.040+0000 7f5813fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:56562/0 (socket says 172.21.15.6:56562) 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 -- 172.21.15.6:0/1746349231 learned_addr learned my addr 172.21.15.6:0/1746349231 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f581959c700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff549140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 -- 172.21.15.6:0/1746349231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 msgr2=0x559bff560aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 0x559bff560aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 -- 172.21.15.6:0/1746349231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 msgr2=0x559bff549140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff549140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 -- 172.21.15.6:0/1746349231 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bff45f130 con 0x559bff54d720 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 0x559bff560aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:13:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f5813fff700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5800096030 tx=0x7f58000b8310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4ca9cb530eb4b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f57f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800095de0 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5800098f10 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.041+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f580000c880 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5800099bc0 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 10 client.26435 fetch_fsmap learned FSMap version 22 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f5808012570 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f580000c880 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5800010c60 con 0x559bff54d720 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 10 client.26435 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:13:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 20 client.26435 populate_metadata read hostname 'smithi006' 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5bd410 0x559bff5bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 4 client.26435 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559bff467600 con 0x559bff5bd410 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.042+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5bd410 0x559bff5bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.043+0000 7f5818d9b700 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5bd410 0x559bff5bf7f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff4856cfcf796f server_cookie=67c5f03cc92819ad in_seq=0 out_seq=0 2023-11-17T00:13:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:42.043+0000 7f57f3fff700 10 client.26435 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:42.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:42 smithi087 ceph-mon[131182]: pgmap v147: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 926 op/s 2023-11-17T00:13:42.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:42 smithi186 ceph-mon[131487]: pgmap v147: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 926 op/s 2023-11-17T00:13:42.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:42 smithi006 ceph-mon[97840]: pgmap v147: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 926 op/s 2023-11-17T00:13:44.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:44 smithi087 ceph-mon[131182]: pgmap v148: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 921 op/s 2023-11-17T00:13:44.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:44 smithi186 ceph-mon[131487]: pgmap v148: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 921 op/s 2023-11-17T00:13:44.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:44 smithi006 ceph-mon[97840]: pgmap v148: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 921 op/s 2023-11-17T00:13:46.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:46 smithi087 ceph-mon[131182]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 1.15k op/s 2023-11-17T00:13:46.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:46 smithi186 ceph-mon[131487]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 1.15k op/s 2023-11-17T00:13:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:46 smithi006 ceph-mon[97840]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 1.15k op/s 2023-11-17T00:13:48.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:48 smithi087 ceph-mon[131182]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:48.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:48 smithi186 ceph-mon[131487]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:48.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:48 smithi006 ceph-mon[97840]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:50.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:50 smithi087 ceph-mon[131182]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:50.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:50 smithi186 ceph-mon[131487]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:50 smithi006 ceph-mon[97840]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.08k op/s 2023-11-17T00:13:52.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:52 smithi087 ceph-mon[131182]: pgmap v152: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 2.37k op/s 2023-11-17T00:13:52.871 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:52 smithi186 ceph-mon[131487]: pgmap v152: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 2.37k op/s 2023-11-17T00:13:52.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:52 smithi006 ceph-mon[97840]: pgmap v152: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 2.37k op/s 2023-11-17T00:13:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.527+0000 7f57f3fff700 1 -- 172.21.15.6:0/1746349231 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559bff467600 con 0x559bff5bd410 2023-11-17T00:13:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f57f3fff700 10 client.26435 handle_command_reply: tid=0 2023-11-17T00:13:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 client.26435 shutdown 2023-11-17T00:13:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 msgr2=0x559bff548c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5800096030 tx=0x7f58000b8310 comp rx=0 tx=0).stop 2023-11-17T00:13:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 shutdown_connections 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559bff5bd410 0x559bff5bf7f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559bff54db90 0x559bff549140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559bff3a36b0 0x559bff560aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 --2- 172.21.15.6:0/1746349231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559bff54d720 0x559bff548c00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.528+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 >> 172.21.15.6:0/1746349231 conn(0x559bff53cf50 msgr2=0x559bff54aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.529+0000 7f57f3fff700 0 client.26435 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.529+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 shutdown_connections 2023-11-17T00:13:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.529+0000 7f582e9b5b80 1 -- 172.21.15.6:0/1746349231 wait complete. 2023-11-17T00:13:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.530+0000 7f582e9b5b80 20 client.26435 trim_cache size 0 max 16384 2023-11-17T00:13:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:13:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:13:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:13:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 msgr2=0x7f58040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 0x7f58040a37f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5814074f00 tx=0x7f580c00b040 comp rx=0 tx=0).stop 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 msgr2=0x7f5814073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814073330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5814101210 tx=0x7f580000eca0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 shutdown_connections 2023-11-17T00:13:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f58040a1340 0x7f58040a37f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58141004a0 0x7f5814072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58141054e0 0x7f58141a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 --2- 172.21.15.6:0/150479062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5814100e40 0x7f5814073330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.532+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 >> 172.21.15.6:0/150479062 conn(0x7f58140fbd10 msgr2=0x7f58140fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.533+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 shutdown_connections 2023-11-17T00:13:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:13:54.533+0000 7f581afff700 1 -- 172.21.15.6:0/150479062 wait complete. 2023-11-17T00:13:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:54 smithi087 ceph-mon[131182]: pgmap v153: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2023-11-17T00:13:54.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:54 smithi186 ceph-mon[131487]: pgmap v153: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2023-11-17T00:13:54.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:54 smithi006 ceph-mon[97840]: pgmap v153: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 1.51k op/s 2023-11-17T00:13:56.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:56 smithi087 ceph-mon[131182]: pgmap v154: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 1.93k op/s 2023-11-17T00:13:56.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:56 smithi186 ceph-mon[131487]: pgmap v154: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 1.93k op/s 2023-11-17T00:13:56.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:56 smithi006 ceph-mon[97840]: pgmap v154: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 1.93k op/s 2023-11-17T00:13:58.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:13:58 smithi087 ceph-mon[131182]: pgmap v155: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:13:58.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:13:58 smithi186 ceph-mon[131487]: pgmap v155: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:13:58.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:13:58 smithi006 ceph-mon[97840]: pgmap v155: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:14:00.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:00 smithi087 ceph-mon[131182]: pgmap v156: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:14:00.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:00 smithi186 ceph-mon[131487]: pgmap v156: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:14:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:00 smithi006 ceph-mon[97840]: pgmap v156: 129 pgs: 129 active+clean; 32 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 1.70k op/s 2023-11-17T00:14:01.043 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 -- 172.21.15.6:0/973360696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc1015f0 msgr2=0x7fbdfc104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 --2- 172.21.15.6:0/973360696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc104ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0099c0 tx=0x7fbdec009cd0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 -- 172.21.15.6:0/973360696 shutdown_connections 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 --2- 172.21.15.6:0/973360696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc10d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 --2- 172.21.15.6:0/973360696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc100c50 0x7fbdfc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 --2- 172.21.15.6:0/973360696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc104ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 -- 172.21.15.6:0/973360696 >> 172.21.15.6:0/973360696 conn(0x7fbdfc077550 msgr2=0x7fbdfc077950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 -- 172.21.15.6:0/973360696 shutdown_connections 2023-11-17T00:14:01.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.276+0000 7fbe039ff700 1 -- 172.21.15.6:0/973360696 wait complete. 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.277+0000 7fbe039ff700 1 Processor -- start 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.277+0000 7fbe039ff700 1 -- start start 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.277+0000 7fbe039ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 0x7fbdfc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.277+0000 7fbe039ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe039ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe039ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a8e00 con 0x7fbdfc1055c0 2023-11-17T00:14:01.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe039ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a8f40 con 0x7fbdfc1015f0 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe039ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a9080 con 0x7fbdfc100c50 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe01f9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44342/0 (socket says 172.21.15.6:44342) 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1577514183 learned_addr learned my addr 172.21.15.6:0/1577514183 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe0179b700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 0x7fbdfc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1577514183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 msgr2=0x7fbdfc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 0x7fbdfc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 msgr2=0x7fbdfc073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1577514183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009520 con 0x7fbdfc1015f0 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe0179b700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 0x7fbdfc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.278+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0098e0 tx=0x7fbdec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.279+0000 7fbdf27fc700 1 -- 172.21.15.6:0/1577514183 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec01d070 con 0x7fbdfc1015f0 2023-11-17T00:14:01.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.279+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec0095f0 con 0x7fbdfc1015f0 2023-11-17T00:14:01.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.279+0000 7fbdf27fc700 1 -- 172.21.15.6:0/1577514183 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec022470 con 0x7fbdfc1015f0 2023-11-17T00:14:01.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.279+0000 7fbdf27fc700 1 -- 172.21.15.6:0/1577514183 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec017650 con 0x7fbdfc1015f0 2023-11-17T00:14:01.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.279+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc1103d0 con 0x7fbdfc1015f0 2023-11-17T00:14:01.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.282+0000 7fbdf27fc700 1 -- 172.21.15.6:0/1577514183 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbdec022850 con 0x7fbdfc1015f0 2023-11-17T00:14:01.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.283+0000 7fbe1727ab80 1 Processor -- start 2023-11-17T00:14:01.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 -- start start 2023-11-17T00:14:01.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbdf27fc700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 0x7fbde80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d2c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4c2d060 con 0x5650d4d28470 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4c7f8a0 con 0x5650d4d2c8b0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4c78d30 con 0x5650d4d27b60 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbdf27fc700 1 -- 172.21.15.6:0/1577514183 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbdec0cdf80 con 0x7fbdfc1015f0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d27f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42050/0 (socket says 172.21.15.6:42050) 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe00f9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d2c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe0179b700 1 -- 172.21.15.6:0/1243672313 learned_addr learned my addr 172.21.15.6:0/1243672313 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.284+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 0x7fbde80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1243672313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 msgr2=0x5650d4d27f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d27f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1243672313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 msgr2=0x5650d4d2cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d2cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 -- 172.21.15.6:0/1243672313 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650d4afa630 con 0x5650d4d28470 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d2c370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0cb320 tx=0x7fbdec09ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7ddc87783bd911 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 0x7fbde80a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009fd0 tx=0x7fbdf400b040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbde77fe700 1 -- 172.21.15.6:0/1243672313 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec09ad30 con 0x5650d4d28470 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbde77fe700 1 -- 172.21.15.6:0/1243672313 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec09ed60 con 0x5650d4d28470 2023-11-17T00:14:01.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.285+0000 7fbde77fe700 1 -- 172.21.15.6:0/1243672313 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec019040 con 0x5650d4d28470 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 -- 172.21.15.6:0/1243672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 msgr2=0x5650d4d2c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d2c370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0cb320 tx=0x7fbdec09ab80 comp rx=0 tx=0).stop 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 -- 172.21.15.6:0/1243672313 shutdown_connections 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d2cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d27f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/1243672313 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d2c370 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 -- 172.21.15.6:0/1243672313 >> 172.21.15.6:0/1243672313 conn(0x5650d4d1d360 msgr2=0x5650d4d22b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 -- 172.21.15.6:0/1243672313 shutdown_connections 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.286+0000 7fbe1727ab80 1 -- 172.21.15.6:0/1243672313 wait complete. 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.287+0000 7fbe1727ab80 1 -- start start 2023-11-17T00:14:01.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.287+0000 7fbe1727ab80 1 Processor -- start 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d33fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4c1a8e0 con 0x5650d4d28470 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4c1a580 con 0x5650d4d2c8b0 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4afc920 con 0x5650d4d27b60 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe1727ab80 10 client.0 _ll_register_callbacks cb 0x5650d4c38780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44346/0 (socket says 172.21.15.6:44346) 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3130491435 learned_addr learned my addr 172.21.15.6:0/3130491435 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d33fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3130491435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 msgr2=0x5650d4d33fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d33fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3130491435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 msgr2=0x5650d4d311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.288+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3130491435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc1103d0 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4003cb0 tx=0x7fbdf4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5494b5368bb8a29c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbde4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021070 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbdf4008e30 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650d4d1ab20 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf400ee60 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbe1727ab80 10 client.26447 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.289+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5650d4c3f190 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650d4d1ab20 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdf402e070 con 0x5650d4d2c8b0 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 10 client.26447 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:01.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 20 client.26447 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d41560 0x5650d4d43970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 4 client.26447 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650d4c3b0a0 con 0x5650d4d41560 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.290+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d41560 0x5650d4d43970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.291+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d41560 0x5650d4d43970 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d33ab1481a906e server_cookie=9fa556b1412a531c in_seq=0 out_seq=0 2023-11-17T00:14:01.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.291+0000 7fbde4ff9700 10 client.26447 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3130491435 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5650d4c3b0a0 con 0x5650d4d41560 2023-11-17T00:14:01.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbde4ff9700 10 client.26447 handle_command_reply: tid=0 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 client.26447 shutdown 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 msgr2=0x5650d4d31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4003cb0 tx=0x7fbdf4004b40 comp rx=0 tx=0).stop 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 shutdown_connections 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d2c8b0 0x5650d4d31700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d33fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d41560 0x5650d4d43970 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3130491435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d28470 0x5650d4d311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbde4ff9700 0 client.26447 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 >> 172.21.15.6:0/3130491435 conn(0x5650d4d1d360 msgr2=0x5650d4d2dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 shutdown_connections 2023-11-17T00:14:01.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.292+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3130491435 wait complete. 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.293+0000 7fbe1727ab80 20 client.26447 trim_cache size 0 max 16384 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.310+0000 7fbe1727ab80 1 Processor -- start 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.310+0000 7fbe1727ab80 1 -- start start 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d2c710 con 0x5650d4d2c8b0 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d24370 con 0x5650d4d28470 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d1cee0 con 0x5650d4d27b60 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44360/0 (socket says 172.21.15.6:44360) 2023-11-17T00:14:01.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 -- 172.21.15.6:0/2461534998 learned_addr learned my addr 172.21.15.6:0/2461534998 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d22b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42064/0 (socket says 172.21.15.6:42064) 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 -- 172.21.15.6:0/2461534998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 msgr2=0x5650d4d22b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d22b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 -- 172.21.15.6:0/2461534998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 msgr2=0x5650d4d3d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d3d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 -- 172.21.15.6:0/2461534998 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650d4d30fd0 con 0x5650d4d28470 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.311+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbdec09ac70 tx=0x7fbdec00b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cda5ec2959f99ce server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbde7fff700 1 -- 172.21.15.6:0/2461534998 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0a30f0 con 0x5650d4d28470 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbde7fff700 1 -- 172.21.15.6:0/2461534998 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec09ad30 con 0x5650d4d28470 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbde7fff700 1 -- 172.21.15.6:0/2461534998 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0cb5a0 con 0x5650d4d28470 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 -- 172.21.15.6:0/2461534998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 msgr2=0x5650d4d387b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbdec09ac70 tx=0x7fbdec00b5b0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 -- 172.21.15.6:0/2461534998 shutdown_connections 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d3d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d27b60 0x5650d4d22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/2461534998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d28470 0x5650d4d387b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 -- 172.21.15.6:0/2461534998 >> 172.21.15.6:0/2461534998 conn(0x5650d4d1d360 msgr2=0x5650d4d3cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 -- 172.21.15.6:0/2461534998 shutdown_connections 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.312+0000 7fbe1727ab80 1 -- 172.21.15.6:0/2461534998 wait complete. 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.313+0000 7fbe1727ab80 1 -- start start 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.313+0000 7fbe1727ab80 1 Processor -- start 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.313+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.313+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d28470 0x5650d4d34cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe1727ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d945e0 con 0x5650d4d2c8b0 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d94750 con 0x5650d4d27b60 2023-11-17T00:14:01.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe1727ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650d4d94910 con 0x5650d4d28470 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe1727ab80 10 client.0 _ll_register_callbacks cb 0x5650d4c38780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe0179b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44372/0 (socket says 172.21.15.6:44372) 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe0179b700 1 -- 172.21.15.6:0/3703353894 learned_addr learned my addr 172.21.15.6:0/3703353894 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe00f9a700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d28470 0x5650d4d34cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3703353894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d28470 msgr2=0x5650d4d34cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d28470 0x5650d4d34cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 msgr2=0x5650d4d34790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe01f9c700 1 -- 172.21.15.6:0/3703353894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650d4afa630 con 0x5650d4d2c8b0 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.314+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:01.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbe01f9c700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d35250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4026da0 tx=0x7fbdf401b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b00149c98c5910 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbde4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:14:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4021040 con 0x5650d4d2c8b0 2023-11-17T00:14:01.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdec09ed60 con 0x5650d4d2c8b0 2023-11-17T00:14:01.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbdf400ee40 con 0x5650d4d2c8b0 2023-11-17T00:14:01.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf4008a60 con 0x5650d4d2c8b0 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbe1727ab80 10 client.16338 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5650d4aa6de0 con 0x5650d4d2c8b0 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.315+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec09ed60 con 0x5650d4d2c8b0 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdf4036070 con 0x5650d4d2c8b0 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 10 client.16338 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 20 client.16338 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d95fa0 0x5650d4d983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 4 client.16338 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650d4c47660 con 0x5650d4d95fa0 2023-11-17T00:14:01.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d95fa0 0x5650d4d983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbde4ff9700 10 client.16338 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.316+0000 7fbe0179b700 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d95fa0 0x5650d4d983b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4e4996fc4cefc8 server_cookie=bd6c32669f141644 in_seq=0 out_seq=0 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbde4ff9700 1 -- 172.21.15.6:0/3703353894 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5650d4c47660 con 0x5650d4d95fa0 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbde4ff9700 10 client.16338 handle_command_reply: tid=0 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 client.16338 shutdown 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 msgr2=0x5650d4d35250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d35250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4026da0 tx=0x7fbdf401b680 comp rx=0 tx=0).stop 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 shutdown_connections 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5650d4d95fa0 0x5650d4d983b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5650d4d2c8b0 0x5650d4d35250 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5650d4d27b60 0x5650d4d34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 --2- 172.21.15.6:0/3703353894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650d4d28470 0x5650d4d34cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbde4ff9700 0 client.16338 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.318+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 >> 172.21.15.6:0/3703353894 conn(0x5650d4d1d360 msgr2=0x5650d4d2bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 shutdown_connections 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe1727ab80 1 -- 172.21.15.6:0/3703353894 wait complete. 2023-11-17T00:14:01.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe1727ab80 20 client.16338 trim_cache size 0 max 16384 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (6334 inodes in the stack)", 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "9ebf54b4-47aa-46ec-acbc-7e7519866a60": { 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:14:01.395 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:14:01.396 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 msgr2=0x7fbde80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 0x7fbde80a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009fd0 tx=0x7fbdf400b040 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 msgr2=0x7fbdfc073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0098e0 tx=0x7fbdec00f690 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 shutdown_connections 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fbde80a12c0 0x7fbde80a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbdfc1055c0 0x7fbdfc073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc100c50 0x7fbdfc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 --2- 172.21.15.6:0/1577514183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdfc1015f0 0x7fbdfc073330 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.319+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 >> 172.21.15.6:0/1577514183 conn(0x7fbdfc077550 msgr2=0x7fbdfc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.320+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 shutdown_connections 2023-11-17T00:14:01.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:01.320+0000 7fbe039ff700 1 -- 172.21.15.6:0/1577514183 wait complete. 2023-11-17T00:14:01.398 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebf54b4-47aa-46ec-acbc-7e7519866a60 - {'path': '/', 'tag': '9ebf54b4-47aa-46ec-acbc-7e7519866a60', 'options': 'recursive,force'} 2023-11-17T00:14:02.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:02 smithi087 ceph-mon[131182]: pgmap v157: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 2k op/s 2023-11-17T00:14:02.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:02 smithi186 ceph-mon[131487]: pgmap v157: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 2k op/s 2023-11-17T00:14:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:02 smithi006 ceph-mon[97840]: pgmap v157: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 2k op/s 2023-11-17T00:14:04.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:04 smithi087 ceph-mon[131182]: pgmap v158: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 715 op/s 2023-11-17T00:14:04.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:04 smithi186 ceph-mon[131487]: pgmap v158: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 715 op/s 2023-11-17T00:14:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:04 smithi006 ceph-mon[97840]: pgmap v158: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 715 op/s 2023-11-17T00:14:06.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:06 smithi087 ceph-mon[131182]: pgmap v159: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.14k op/s 2023-11-17T00:14:06.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:06 smithi186 ceph-mon[131487]: pgmap v159: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.14k op/s 2023-11-17T00:14:06.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:06 smithi006 ceph-mon[97840]: pgmap v159: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.14k op/s 2023-11-17T00:14:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 -- 172.21.15.6:0/233998261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a01004c0 msgr2=0x7f59a0100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 --2- 172.21.15.6:0/233998261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a01004c0 0x7f59a0100890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5994009990 tx=0x7f5994009ca0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 -- 172.21.15.6:0/233998261 shutdown_connections 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 --2- 172.21.15.6:0/233998261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a010d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 --2- 172.21.15.6:0/233998261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a01004c0 0x7f59a0100890 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 --2- 172.21.15.6:0/233998261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a0100e60 0x7f59a0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 -- 172.21.15.6:0/233998261 >> 172.21.15.6:0/233998261 conn(0x7f59a00fbd10 msgr2=0x7f59a00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 -- 172.21.15.6:0/233998261 shutdown_connections 2023-11-17T00:14:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.709+0000 7f59a6dff700 1 -- 172.21.15.6:0/233998261 wait complete. 2023-11-17T00:14:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.710+0000 7f59a6dff700 1 Processor -- start 2023-11-17T00:14:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.710+0000 7f59a6dff700 1 -- start start 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a01004c0 0x7f59a0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a0073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a47d0 con 0x7f59a0105500 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a4940 con 0x7f59a0100e60 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a6dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01a4b00 con 0x7f59a01004c0 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44392/0 (socket says 172.21.15.6:44392) 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f599ffff700 1 -- 172.21.15.6:0/3474533769 learned_addr learned my addr 172.21.15.6:0/3474533769 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a01004c0 0x7f59a0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.711+0000 7f59a539c700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a0073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 -- 172.21.15.6:0/3474533769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a01004c0 msgr2=0x7f59a0072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a01004c0 0x7f59a0072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 -- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 msgr2=0x7f59a0073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a0073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 -- 172.21.15.6:0/3474533769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940095f0 con 0x7f59a0100e60 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f59a539c700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a0073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599ffff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599dffb700 1 -- 172.21.15.6:0/3474533769 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c004220 con 0x7f59a0100e60 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599dffb700 1 -- 172.21.15.6:0/3474533769 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c004cf0 con 0x7f59a0100e60 2023-11-17T00:14:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a0110310 con 0x7f59a0100e60 2023-11-17T00:14:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.712+0000 7f599dffb700 1 -- 172.21.15.6:0/3474533769 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c005620 con 0x7f59a0100e60 2023-11-17T00:14:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.713+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a0110180 con 0x7f59a0100e60 2023-11-17T00:14:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.716+0000 7f599dffb700 1 -- 172.21.15.6:0/3474533769 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f598c004750 con 0x7f59a0100e60 2023-11-17T00:14:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 Processor -- start 2023-11-17T00:14:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 -- start start 2023-11-17T00:14:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f599dffb700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 0x7f59900a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab6f4b90 0x5642ab6eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab5f4000 con 0x5642ab6f4b90 2023-11-17T00:14:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f599dffb700 1 -- 172.21.15.6:0/3474533769 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f598c0cce40 con 0x7f59a0100e60 2023-11-17T00:14:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab646840 con 0x5642ab6f4720 2023-11-17T00:14:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.718+0000 7f59ba637b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab63fcd0 con 0x5642ab54a6b0 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42102/0 (socket says 172.21.15.6:42102) 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a539c700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 0x7f59900a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 -- 172.21.15.6:0/1385147168 learned_addr learned my addr 172.21.15.6:0/1385147168 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f599ffff700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a539c700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab6f4b90 0x5642ab6eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 -- 172.21.15.6:0/1385147168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 msgr2=0x5642ab6ee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6ee560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 -- 172.21.15.6:0/1385147168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab6f4b90 msgr2=0x5642ab6eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab6f4b90 0x5642ab6eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f59a4b9b700 1 -- 172.21.15.6:0/1385147168 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ab4c15d0 con 0x5642ab54a6b0 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.719+0000 7f599ffff700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.720+0000 7f59a539c700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 0x7f59900a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5998015e20 tx=0x7f5998015420 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.720+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5994005f50 tx=0x7f599400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e41fbb482e20a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.720+0000 7f598affd700 1 -- 172.21.15.6:0/1385147168 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599401d070 con 0x5642ab54a6b0 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.720+0000 7f598affd700 1 -- 172.21.15.6:0/1385147168 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599401e040 con 0x5642ab54a6b0 2023-11-17T00:14:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.720+0000 7f598affd700 1 -- 172.21.15.6:0/1385147168 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994004880 con 0x5642ab54a6b0 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 -- 172.21.15.6:0/1385147168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 msgr2=0x5642ab6f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5994005f50 tx=0x7f599400b610 comp rx=0 tx=0).stop 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 -- 172.21.15.6:0/1385147168 shutdown_connections 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab6f4b90 0x5642ab6eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab54a6b0 0x5642ab6f41e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1385147168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 -- 172.21.15.6:0/1385147168 >> 172.21.15.6:0/1385147168 conn(0x5642ab6e3f50 msgr2=0x5642ab6e9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 -- 172.21.15.6:0/1385147168 shutdown_connections 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.721+0000 7f59ba637b80 1 -- 172.21.15.6:0/1385147168 wait complete. 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.722+0000 7f59ba637b80 1 -- start start 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.722+0000 7f59ba637b80 1 Processor -- start 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6f8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab5e1880 con 0x5642ab54a6b0 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab5e1520 con 0x5642ab6f4720 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab4c38c0 con 0x5642ab6f4b90 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59ba637b80 10 client.0 _ll_register_callbacks cb 0x5642ab5ff720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44418/0 (socket says 172.21.15.6:44418) 2023-11-17T00:14:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f599ffff700 1 -- 172.21.15.6:0/1257788289 learned_addr learned my addr 172.21.15.6:0/1257788289 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.723+0000 7f59a539c700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 -- 172.21.15.6:0/1257788289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 msgr2=0x5642ab6f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 -- 172.21.15.6:0/1257788289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 msgr2=0x5642ab6faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 -- 172.21.15.6:0/1257788289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a010d910 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f59a539c700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f599ffff700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f598c099860 tx=0x7f598c099c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64876af55ae1f92e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f597ffff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c004380 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f598c0b9040 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642ab6e1710 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c0b2200 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f59ba637b80 10 client.26465 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.724+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5642ab606130 con 0x5642ab6f4720 2023-11-17T00:14:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642ab6e1710 con 0x5642ab6f4720 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f598c099050 con 0x5642ab6f4720 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 20 client.26465 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab707ee0 0x5642ab70a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 4 client.26465 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642ab602040 con 0x5642ab707ee0 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.725+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab707ee0 0x5642ab70a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.726+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab707ee0 0x5642ab70a2f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86773b1d61b942da server_cookie=3c3496fd50781ed5 in_seq=0 out_seq=0 2023-11-17T00:14:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.726+0000 7f597ffff700 10 client.26465 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f597ffff700 1 -- 172.21.15.6:0/1257788289 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642ab602040 con 0x5642ab707ee0 2023-11-17T00:14:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f597ffff700 10 client.26465 handle_command_reply: tid=0 2023-11-17T00:14:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f59ba637b80 1 client.26465 shutdown 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 msgr2=0x5642ab6f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f598c099860 tx=0x7f598c099c50 comp rx=0 tx=0).stop 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 shutdown_connections 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.730+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab707ee0 0x5642ab70a2f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f597ffff700 0 client.26465 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 --2- 172.21.15.6:0/1257788289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6f7c40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 >> 172.21.15.6:0/1257788289 conn(0x5642ab6e3f50 msgr2=0x5642ab6e87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 shutdown_connections 2023-11-17T00:14:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 1 -- 172.21.15.6:0/1257788289 wait complete. 2023-11-17T00:14:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.731+0000 7f59ba637b80 20 client.26465 trim_cache size 0 max 16384 2023-11-17T00:14:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.748+0000 7f59ba637b80 1 Processor -- start 2023-11-17T00:14:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.748+0000 7f59ba637b80 1 -- start start 2023-11-17T00:14:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.748+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.748+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59ba637b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6f41e0 con 0x5642ab54a6b0 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59ba637b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6ea1c0 con 0x5642ab6f4720 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59ba637b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6f3df0 con 0x5642ab6f4b90 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:44432/0 (socket says 172.21.15.6:44432) 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 -- 172.21.15.6:0/3832206178 learned_addr learned my addr 172.21.15.6:0/3832206178 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59a539c700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 -- 172.21.15.6:0/3832206178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 msgr2=0x5642ab6fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 -- 172.21.15.6:0/3832206178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 msgr2=0x5642ab6ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 -- 172.21.15.6:0/3832206178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ab605e80 con 0x5642ab6f4720 2023-11-17T00:14:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f59a539c700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:14:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.749+0000 7f599ffff700 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f598c097eb0 tx=0x7f598c0a2130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b255ee9d8628e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.750+0000 7f598b7fe700 1 -- 172.21.15.6:0/3832206178 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c014070 con 0x5642ab6f4720 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.750+0000 7f598b7fe700 1 -- 172.21.15.6:0/3832206178 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c0b9650 con 0x5642ab6f4720 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.750+0000 7f598b7fe700 1 -- 172.21.15.6:0/3832206178 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a1d60 con 0x5642ab6f4720 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- 172.21.15.6:0/3832206178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 msgr2=0x5642ab6e8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f598c097eb0 tx=0x7f598c0a2130 comp rx=0 tx=0).stop 2023-11-17T00:14:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- 172.21.15.6:0/3832206178 shutdown_connections 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 --2- 172.21.15.6:0/3832206178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6f4720 0x5642ab6e8f60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- 172.21.15.6:0/3832206178 >> 172.21.15.6:0/3832206178 conn(0x5642ab6e3f50 msgr2=0x5642ab6f1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- 172.21.15.6:0/3832206178 shutdown_connections 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- 172.21.15.6:0/3832206178 wait complete. 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.751+0000 7f59ba637b80 1 -- start start 2023-11-17T00:14:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.752+0000 7f59ba637b80 1 Processor -- start 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.752+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.752+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59ba637b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6edbf0 0x5642ab6ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59ba637b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6ee770 con 0x5642ab54a6b0 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59ba637b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6ee8e0 con 0x5642ab6edbf0 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59ba637b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ab6eeaa0 con 0x5642ab6f4b90 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59ba637b80 10 client.0 _ll_register_callbacks cb 0x7f598c0996a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49148/0 (socket says 172.21.15.6:49148) 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 -- 172.21.15.6:0/2663240677 learned_addr learned my addr 172.21.15.6:0/2663240677 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f599ffff700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a539c700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6edbf0 0x5642ab6ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 -- 172.21.15.6:0/2663240677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 msgr2=0x5642ab6ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 -- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6edbf0 msgr2=0x5642ab6ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6edbf0 0x5642ab6ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.753+0000 7f59a4b9b700 1 -- 172.21.15.6:0/2663240677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ab606130 con 0x5642ab54a6b0 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f59a4b9b700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5994009ce0 tx=0x7f599400b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c2c8b93905576e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f597ffff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994031040 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f59940334b0 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f599401e040 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599401d040 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.754+0000 7f59ba637b80 10 client.16356 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5642ab456c00 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599401e040 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5994033d20 con 0x5642ab54a6b0 2023-11-17T00:14:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 20 client.16356 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab75c560 0x5642ab75e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 4 client.16356 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642ab60e600 con 0x5642ab75c560 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.755+0000 7f599ffff700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab75c560 0x5642ab75e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.756+0000 7f597ffff700 10 client.16356 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:07.756+0000 7f599ffff700 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab75c560 0x5642ab75e970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7121ebe40f859832 server_cookie=2b7251750ebb1aaf in_seq=0 out_seq=0 2023-11-17T00:14:08.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:08 smithi087 ceph-mon[131182]: pgmap v160: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:08.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:08 smithi186 ceph-mon[131487]: pgmap v160: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:08 smithi006 ceph-mon[97840]: pgmap v160: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:10.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:10 smithi087 ceph-mon[131182]: pgmap v161: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:10.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:10 smithi186 ceph-mon[131487]: pgmap v161: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:10.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:10 smithi006 ceph-mon[97840]: pgmap v161: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 724 op/s 2023-11-17T00:14:12.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:12 smithi087 ceph-mon[131182]: pgmap v162: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 858 op/s 2023-11-17T00:14:12.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:12 smithi186 ceph-mon[131487]: pgmap v162: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 858 op/s 2023-11-17T00:14:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:12 smithi006 ceph-mon[97840]: pgmap v162: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 858 op/s 2023-11-17T00:14:14.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:14 smithi087 ceph-mon[131182]: pgmap v163: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 562 op/s 2023-11-17T00:14:14.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:14 smithi186 ceph-mon[131487]: pgmap v163: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 562 op/s 2023-11-17T00:14:14.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:14 smithi006 ceph-mon[97840]: pgmap v163: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 562 op/s 2023-11-17T00:14:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.822+0000 7f597ffff700 1 -- 172.21.15.6:0/2663240677 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642ab60e600 con 0x5642ab75c560 2023-11-17T00:14:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.822+0000 7f597ffff700 10 client.16356 handle_command_reply: tid=0 2023-11-17T00:14:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.822+0000 7f59ba637b80 1 client.16356 shutdown 2023-11-17T00:14:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.824+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 msgr2=0x5642ab6ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.824+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5994009ce0 tx=0x7f599400b280 comp rx=0 tx=0).stop 2023-11-17T00:14:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.825+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 shutdown_connections 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.825+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5642ab75c560 0x5642ab75e970 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.826+0000 7f597ffff700 0 client.16356 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.826+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5642ab54a6b0 0x5642ab6ff230 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.826+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5642ab6edbf0 0x5642ab6ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.826+0000 7f59ba637b80 1 --2- 172.21.15.6:0/2663240677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5642ab6f4b90 0x5642ab6ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.826+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 >> 172.21.15.6:0/2663240677 conn(0x5642ab6e3f50 msgr2=0x5642ab6f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.829+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 shutdown_connections 2023-11-17T00:14:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.830+0000 7f59ba637b80 1 -- 172.21.15.6:0/2663240677 wait complete. 2023-11-17T00:14:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.830+0000 7f59ba637b80 20 client.16356 trim_cache size 0 max 16384 2023-11-17T00:14:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:14:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:14:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:14:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:14:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.831+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 msgr2=0x7f59900a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.831+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 0x7f59900a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5998015e20 tx=0x7f5998015420 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.831+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 msgr2=0x7f59a0073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.831+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00dc60 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 shutdown_connections 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59900a1340 0x7f59900a37f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59a0105500 0x7f59a0073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59a01004c0 0x7f59a0072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 --2- 172.21.15.6:0/3474533769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0100e60 0x7f59a0073330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.832+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 >> 172.21.15.6:0/3474533769 conn(0x7f59a00fbd10 msgr2=0x7f59a00fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.834+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 shutdown_connections 2023-11-17T00:14:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:16.834+0000 7f59a6dff700 1 -- 172.21.15.6:0/3474533769 wait complete. 2023-11-17T00:14:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:16 smithi087 ceph-mon[131182]: pgmap v164: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.63k op/s 2023-11-17T00:14:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:16 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:14:16.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:16 smithi186 ceph-mon[131487]: pgmap v164: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.63k op/s 2023-11-17T00:14:16.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:16 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:14:16.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:16 smithi006 ceph-mon[97840]: pgmap v164: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.63k op/s 2023-11-17T00:14:16.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:16 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:14:18.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:18 smithi186 ceph-mon[131487]: pgmap v165: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:18.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:18 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:14:18.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:18 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:14:18.869 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:18 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:14:18.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:18 smithi006 ceph-mon[97840]: pgmap v165: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:18.976 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:18 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:14:18.977 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:18 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:14:18.977 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:18 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:14:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:18 smithi087 ceph-mon[131182]: pgmap v165: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:18 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:14:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:18 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:14:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:18 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:14:19.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:19 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:14:19.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:19 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:14:19.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:19 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:14:19.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:19 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:14:19.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:19 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:14:19.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:19 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:14:19.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:19 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:14:19.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:19 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:14:20.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:19 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:14:20.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:19 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:14:20.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:19 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:14:20.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:19 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:14:20.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:20 smithi186 ceph-mon[131487]: pgmap v166: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:20 smithi006 ceph-mon[97840]: pgmap v166: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:20 smithi087 ceph-mon[131182]: pgmap v166: 129 pgs: 129 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-11-17T00:14:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:22 smithi006 ceph-mon[97840]: pgmap v167: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2023-11-17T00:14:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:22 smithi087 ceph-mon[131182]: pgmap v167: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2023-11-17T00:14:23.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:22 smithi186 ceph-mon[131487]: pgmap v167: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 1.43k op/s 2023-11-17T00:14:24.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:24 smithi006 ceph-mon[97840]: pgmap v168: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-17T00:14:25.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:24 smithi087 ceph-mon[131182]: pgmap v168: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-17T00:14:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:24 smithi186 ceph-mon[131487]: pgmap v168: 129 pgs: 129 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2023-11-17T00:14:26.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:26 smithi006 ceph-mon[97840]: pgmap v169: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2023-11-17T00:14:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:26 smithi087 ceph-mon[131182]: pgmap v169: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2023-11-17T00:14:27.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:26 smithi186 ceph-mon[131487]: pgmap v169: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.69k op/s 2023-11-17T00:14:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:28 smithi006 ceph-mon[97840]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:28 smithi087 ceph-mon[131182]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:28 smithi186 ceph-mon[131487]: pgmap v170: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1117350662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 msgr2=0x7f0d78110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1117350662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d78110d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009990 tx=0x7f0d68009ca0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1117350662 shutdown_connections 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1117350662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78111240 0x7f0d78113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1117350662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78072e90 0x7f0d78073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1117350662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d78110d00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1117350662 >> 172.21.15.6:0/1117350662 conn(0x7f0d780fbd10 msgr2=0x7f0d780fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.116+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1117350662 shutdown_connections 2023-11-17T00:14:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.117+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1117350662 wait complete. 2023-11-17T00:14:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.119+0000 7f0d7fdff700 1 Processor -- start 2023-11-17T00:14:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.119+0000 7f0d7fdff700 1 -- start start 2023-11-17T00:14:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.119+0000 7f0d7fdff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78072e90 0x7f0d781a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.119+0000 7f0d7fdff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78111240 0x7f0d7819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7fdff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7819ce70 con 0x7f0d78072e90 2023-11-17T00:14:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7fdff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7819cfe0 con 0x7f0d780737a0 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7819d1a0 con 0x7f0d78111240 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7d39a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7d39a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43888/0 (socket says 172.21.15.6:43888) 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7d39a700 1 -- 172.21.15.6:0/1752554399 learned_addr learned my addr 172.21.15.6:0/1752554399 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.120+0000 7f0d7e39c700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78111240 0x7f0d7819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7d39a700 1 -- 172.21.15.6:0/1752554399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78111240 msgr2=0x7f0d7819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78111240 0x7f0d7819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7d39a700 1 -- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78072e90 msgr2=0x7f0d781a2790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:14:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78072e90 0x7f0d781a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78072e90 0x7f0d781a2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.121+0000 7f0d7d39a700 1 -- 172.21.15.6:0/1752554399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680095f0 con 0x7f0d780737a0 2023-11-17T00:14:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.124+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d68005910 tx=0x7f0d680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.124+0000 7f0d6effd700 1 -- 172.21.15.6:0/1752554399 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6801d070 con 0x7f0d780737a0 2023-11-17T00:14:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.125+0000 7f0d6effd700 1 -- 172.21.15.6:0/1752554399 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d68022470 con 0x7f0d780737a0 2023-11-17T00:14:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.125+0000 7f0d6effd700 1 -- 172.21.15.6:0/1752554399 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68017650 con 0x7f0d780737a0 2023-11-17T00:14:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.125+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d781168d0 con 0x7f0d780737a0 2023-11-17T00:14:30.128 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.125+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d78116740 con 0x7f0d780737a0 2023-11-17T00:14:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.132+0000 7f0d6effd700 1 -- 172.21.15.6:0/1752554399 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0d680225e0 con 0x7f0d780737a0 2023-11-17T00:14:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.136+0000 7f0d6effd700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 0x7f0d640a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.137+0000 7f0d6effd700 1 -- 172.21.15.6:0/1752554399 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0d680ce3b0 con 0x7f0d780737a0 2023-11-17T00:14:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.137+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 0x7f0d640a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.138+0000 7f0d937efb80 1 Processor -- start 2023-11-17T00:14:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.139+0000 7f0d937efb80 1 -- start start 2023-11-17T00:14:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.139+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 0x7f0d640a38c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009b30 tx=0x7f0d74009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.139+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e7721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d937efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e672000 con 0x55974e5c86b0 2023-11-17T00:14:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d937efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e6c4840 con 0x55974e772b90 2023-11-17T00:14:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d937efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e6bdcd0 con 0x55974e772720 2023-11-17T00:14:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e7721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d7e39c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d7d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d7d39a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:52720/0 (socket says 172.21.15.6:52720) 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.140+0000 7f0d7d39a700 1 -- 172.21.15.6:0/248547108 learned_addr learned my addr 172.21.15.6:0/248547108 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 -- 172.21.15.6:0/248547108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 msgr2=0x55974e76cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 -- 172.21.15.6:0/248547108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 msgr2=0x55974e7721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e7721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 -- 172.21.15.6:0/248547108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974e53f5d0 con 0x55974e772720 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e7721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.141+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d680a2ff0 tx=0x7f0d6809ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff8ad1ffe7de82 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.142+0000 7f0d63fff700 1 -- 172.21.15.6:0/248547108 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d680cbea0 con 0x55974e772720 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.142+0000 7f0d63fff700 1 -- 172.21.15.6:0/248547108 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6809f190 con 0x55974e772720 2023-11-17T00:14:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.142+0000 7f0d63fff700 1 -- 172.21.15.6:0/248547108 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6809b620 con 0x55974e772720 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.142+0000 7f0d937efb80 1 -- 172.21.15.6:0/248547108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 msgr2=0x55974e76c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.142+0000 7f0d937efb80 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d680a2ff0 tx=0x7f0d6809ad20 comp rx=0 tx=0).stop 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 -- 172.21.15.6:0/248547108 shutdown_connections 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e7721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 --2- 172.21.15.6:0/248547108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772720 0x55974e76c560 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 -- 172.21.15.6:0/248547108 >> 172.21.15.6:0/248547108 conn(0x55974e761f50 msgr2=0x55974e767270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 -- 172.21.15.6:0/248547108 shutdown_connections 2023-11-17T00:14:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 -- 172.21.15.6:0/248547108 wait complete. 2023-11-17T00:14:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.143+0000 7f0d937efb80 1 -- start start 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 Processor -- start 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772720 0x55974e77bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e775a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e65f880 con 0x55974e5c86b0 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e65f520 con 0x55974e772720 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e5418c0 con 0x55974e772b90 2023-11-17T00:14:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.144+0000 7f0d937efb80 10 client.0 _ll_register_callbacks cb 0x55974e67d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7e39c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e775a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45230/0 (socket says 172.21.15.6:45230) 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 -- 172.21.15.6:0/4005869618 learned_addr learned my addr 172.21.15.6:0/4005869618 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772720 0x55974e77bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 -- 172.21.15.6:0/4005869618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 msgr2=0x55974e775a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e775a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 -- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772720 msgr2=0x55974e77bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772720 0x55974e77bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.145+0000 7f0d7db9b700 1 -- 172.21.15.6:0/4005869618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d78116740 con 0x55974e5c86b0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.146+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d74010d90 tx=0x7f0d74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f840d45aef91767 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.146+0000 7f0d617fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.146+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74012ae0 con 0x55974e5c86b0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.146+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0d74012c40 con 0x55974e5c86b0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.146+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55974e75f710 con 0x55974e5c86b0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.147+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d74012e00 con 0x55974e5c86b0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.147+0000 7f0d937efb80 10 client.16362 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.147+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55974e684130 con 0x55974e5c86b0 2023-11-17T00:14:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.148+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55974e75f710 con 0x55974e5c86b0 2023-11-17T00:14:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.148+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d740315e0 con 0x55974e5c86b0 2023-11-17T00:14:30.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 10 client.16362 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 20 client.16362 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e785b30 0x55974e787f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 4 client.16362 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55974e680040 con 0x55974e785b30 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e785b30 0x55974e787f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e785b30 0x55974e787f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287603da161657d1 server_cookie=3ed06a38ce9aa335 in_seq=0 out_seq=0 2023-11-17T00:14:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.149+0000 7f0d617fa700 10 client.16362 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d617fa700 1 -- 172.21.15.6:0/4005869618 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55974e680040 con 0x55974e785b30 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d617fa700 10 client.16362 handle_command_reply: tid=0 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 client.16362 shutdown 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 msgr2=0x55974e77b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d74010d90 tx=0x7f0d74004750 comp rx=0 tx=0).stop 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 shutdown_connections 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e785b30 0x55974e787f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e5c86b0 0x55974e77b720 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e775a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 --2- 172.21.15.6:0/4005869618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772720 0x55974e77bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 >> 172.21.15.6:0/4005869618 conn(0x55974e761f50 msgr2=0x55974e7667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d617fa700 0 client.16362 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 shutdown_connections 2023-11-17T00:14:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.153+0000 7f0d937efb80 1 -- 172.21.15.6:0/4005869618 wait complete. 2023-11-17T00:14:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.154+0000 7f0d937efb80 20 client.16362 trim_cache size 0 max 16384 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.171+0000 7f0d937efb80 1 Processor -- start 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 -- start start 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e766f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e761ba0 con 0x55974e772720 2023-11-17T00:14:30.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e771df0 con 0x55974e772b90 2023-11-17T00:14:30.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d937efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e768c50 con 0x55974e5c86b0 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:52726/0 (socket says 172.21.15.6:52726) 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d7db9b700 1 -- 172.21.15.6:0/2819475983 learned_addr learned my addr 172.21.15.6:0/2819475983 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.172+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e766f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7e39c700 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 -- 172.21.15.6:0/2819475983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 msgr2=0x55974e76ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 -- 172.21.15.6:0/2819475983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 msgr2=0x55974e766f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e766f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 -- 172.21.15.6:0/2819475983 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974e683e80 con 0x55974e5c86b0 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d74028fd0 tx=0x7f0d74031f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21662a76ef98f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.173+0000 7f0d6cbff700 1 -- 172.21.15.6:0/2819475983 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74031420 con 0x55974e5c86b0 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.174+0000 7f0d6cbff700 1 -- 172.21.15.6:0/2819475983 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d74004cf0 con 0x55974e5c86b0 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.174+0000 7f0d6cbff700 1 -- 172.21.15.6:0/2819475983 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74005580 con 0x55974e5c86b0 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.174+0000 7f0d937efb80 1 -- 172.21.15.6:0/2819475983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 msgr2=0x55974e7736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.174+0000 7f0d937efb80 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d74028fd0 tx=0x7f0d74031f90 comp rx=0 tx=0).stop 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 -- 172.21.15.6:0/2819475983 shutdown_connections 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e772b90 0x55974e76ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e5c86b0 0x55974e7736a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 --2- 172.21.15.6:0/2819475983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e766f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 -- 172.21.15.6:0/2819475983 >> 172.21.15.6:0/2819475983 conn(0x55974e761f50 msgr2=0x55974e772f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 -- 172.21.15.6:0/2819475983 shutdown_connections 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 -- 172.21.15.6:0/2819475983 wait complete. 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 -- start start 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.175+0000 7f0d937efb80 1 Processor -- start 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e779040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e779780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e779e90 con 0x55974e772720 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e7d8990 con 0x55974e5c86b0 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e7d8ad0 con 0x55974e772b90 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d937efb80 10 client.0 _ll_register_callbacks cb 0x7f0d74010970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d7db9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43930/0 (socket says 172.21.15.6:43930) 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.176+0000 7f0d7db9b700 1 -- 172.21.15.6:0/3851473599 learned_addr learned my addr 172.21.15.6:0/3851473599 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7e39c700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e779780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 -- 172.21.15.6:0/3851473599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 msgr2=0x55974e779780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e779780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 -- 172.21.15.6:0/3851473599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 msgr2=0x55974e779040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e779040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e779040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 -- 172.21.15.6:0/3851473599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55974e684130 con 0x55974e5c86b0 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e779040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d7db9b700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d74033010 tx=0x7f0d740092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6a8d5b9b245911 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.177+0000 7f0d617fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:14:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74004940 con 0x55974e5c86b0 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0d74004aa0 con 0x55974e5c86b0 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d6809f190 con 0x55974e5c86b0 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d74005200 con 0x55974e5c86b0 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d937efb80 10 client.26489 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55974e4d4c00 con 0x55974e5c86b0 2023-11-17T00:14:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.178+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6809f190 con 0x55974e5c86b0 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d7401e410 con 0x55974e5c86b0 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 10 client.26489 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 20 client.26489 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e7d9d50 0x55974e7dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 4 client.26489 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55974e68c600 con 0x55974e7d9d50 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.179+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e7d9d50 0x55974e7dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.180+0000 7f0d7d39a700 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e7d9d50 0x55974e7dc130 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef797c98e3e41701 server_cookie=64f5759d4a25062f in_seq=0 out_seq=0 2023-11-17T00:14:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:30.180+0000 7f0d617fa700 10 client.26489 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:30.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:30 smithi006 ceph-mon[97840]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:30 smithi087 ceph-mon[131182]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:30 smithi186 ceph-mon[131487]: pgmap v171: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 627 op/s 2023-11-17T00:14:31.398 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:14:31.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 -- 172.21.15.6:0/3182235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 msgr2=0x7fd02010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 --2- 172.21.15.6:0/3182235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd02010d0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd01800b0a0 tx=0x7fd01800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 -- 172.21.15.6:0/3182235475 shutdown_connections 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 --2- 172.21.15.6:0/3182235475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd02010d0e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 --2- 172.21.15.6:0/3182235475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020105250 0x7fd020105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 --2- 172.21.15.6:0/3182235475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020067dc0 0x7fd020068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.796+0000 7fd025fff700 1 -- 172.21.15.6:0/3182235475 >> 172.21.15.6:0/3182235475 conn(0x7fd0200763a0 msgr2=0x7fd0200767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.797+0000 7fd025fff700 1 -- 172.21.15.6:0/3182235475 shutdown_connections 2023-11-17T00:14:31.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.797+0000 7fd025fff700 1 -- 172.21.15.6:0/3182235475 wait complete. 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.798+0000 7fd025fff700 1 Processor -- start 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 -- start start 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd020198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 0x7fd02019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019d620 con 0x7fd020105250 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019d790 con 0x7fd020067dc0 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd025fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019d950 con 0x7fd020068770 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.799+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.800+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43944/0 (socket says 172.21.15.6:43944) 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.800+0000 7fd01f7fe700 1 -- 172.21.15.6:0/1882546371 learned_addr learned my addr 172.21.15.6:0/1882546371 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:31.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.800+0000 7fd01effd700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd020198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.800+0000 7fd01ffff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 0x7fd02019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 -- 172.21.15.6:0/1882546371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 msgr2=0x7fd020198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd020198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 -- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 msgr2=0x7fd02019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:31.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 0x7fd02019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 -- 172.21.15.6:0/1882546371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009d00 con 0x7fd020067dc0 2023-11-17T00:14:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01effd700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd020198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:14:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01ffff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 0x7fd02019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:14:31.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd01000c920 tx=0x7fd01000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.801+0000 7fd01cff9700 1 -- 172.21.15.6:0/1882546371 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010007b00 con 0x7fd020067dc0 2023-11-17T00:14:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.802+0000 7fd01cff9700 1 -- 172.21.15.6:0/1882546371 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd010004cf0 con 0x7fd020067dc0 2023-11-17T00:14:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.802+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd020110470 con 0x7fd020067dc0 2023-11-17T00:14:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.802+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0201102e0 con 0x7fd020067dc0 2023-11-17T00:14:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.802+0000 7fd01cff9700 1 -- 172.21.15.6:0/1882546371 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010005610 con 0x7fd020067dc0 2023-11-17T00:14:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.807+0000 7fd0398f2b80 1 Processor -- start 2023-11-17T00:14:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.807+0000 7fd0398f2b80 1 -- start start 2023-11-17T00:14:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108ca97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108ca9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c99d060 con 0x56108ca97b60 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c9ef8a0 con 0x56108ca98470 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.808+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c9e8d30 con 0x56108ca9c8b0 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108ca97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:52768/0 (socket says 172.21.15.6:52768) 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 -- 172.21.15.6:0/3761703517 learned_addr learned my addr 172.21.15.6:0/3761703517 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108ca97f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45278/0 (socket says 172.21.15.6:45278) 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 -- 172.21.15.6:0/3761703517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 msgr2=0x56108ca9c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108ca9c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 -- 172.21.15.6:0/3761703517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 msgr2=0x56108ca97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:31.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108ca97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.809+0000 7fd01effd700 1 -- 172.21.15.6:0/3761703517 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108c86a630 con 0x56108ca9c8b0 2023-11-17T00:14:31.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.810+0000 7fd01effd700 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d80 tx=0x7fd0140116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67eb7dcf0b74fdad server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:31.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.810+0000 7fd01cff9700 1 -- 172.21.15.6:0/1882546371 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd010004750 con 0x7fd020067dc0 2023-11-17T00:14:31.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.811+0000 7fd005ffb700 1 -- 172.21.15.6:0/3761703517 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014011e20 con 0x56108ca9c8b0 2023-11-17T00:14:31.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.811+0000 7fd005ffb700 1 -- 172.21.15.6:0/3761703517 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd014012910 con 0x56108ca9c8b0 2023-11-17T00:14:31.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.811+0000 7fd005ffb700 1 -- 172.21.15.6:0/3761703517 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400f860 con 0x56108ca9c8b0 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 -- 172.21.15.6:0/3761703517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 msgr2=0x56108ca9cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d80 tx=0x7fd0140116c0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 -- 172.21.15.6:0/3761703517 shutdown_connections 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108ca97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108ca9cd00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/3761703517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108ca9c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.812+0000 7fd0398f2b80 1 -- 172.21.15.6:0/3761703517 >> 172.21.15.6:0/3761703517 conn(0x56108ca8d360 msgr2=0x56108ca92b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.813+0000 7fd0398f2b80 1 -- 172.21.15.6:0/3761703517 shutdown_connections 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.813+0000 7fd0398f2b80 1 -- 172.21.15.6:0/3761703517 wait complete. 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.813+0000 7fd01cff9700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 0x7fd0080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.813+0000 7fd01cff9700 1 -- 172.21.15.6:0/1882546371 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd0100cd020 con 0x7fd020067dc0 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.814+0000 7fd0398f2b80 1 -- start start 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.814+0000 7fd0398f2b80 1 Processor -- start 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.815+0000 7fd01effd700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 0x7fd0080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd01effd700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 0x7fd0080a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd01400a010 tx=0x7fd014012f50 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca97b60 0x56108caa1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca98470 0x56108caa18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c98a8e0 con 0x56108ca98470 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c98a580 con 0x56108ca97b60 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108c86c920 con 0x56108ca9c8b0 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.816+0000 7fd0398f2b80 10 client.0 _ll_register_callbacks cb 0x56108c9a8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.817+0000 7fd01effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca98470 0x56108caa18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.817+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.817+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:52776/0 (socket says 172.21.15.6:52776) 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.817+0000 7fd01ffff700 1 -- 172.21.15.6:0/974083633 learned_addr learned my addr 172.21.15.6:0/974083633 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.817+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca97b60 0x56108caa1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 -- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca97b60 msgr2=0x56108caa1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca97b60 0x56108caa1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 -- 172.21.15.6:0/974083633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca98470 msgr2=0x56108caa18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca98470 0x56108caa18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:31.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 -- 172.21.15.6:0/974083633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0140099b0 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fd01ffff700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd020105e80 tx=0x7fd01800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43fcbc1bfb52006e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.818+0000 7fcffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.819+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800e040 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.819+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd018022070 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.819+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56108ca8ab20 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.820+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0180074e0 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.821+0000 7fd0398f2b80 10 client.26467 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.821+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56108c9af190 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.821+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56108ca8ab20 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.821+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd01800b0d0 con 0x56108ca9c8b0 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:31.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 20 client.26467 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:31.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108caba1d0 0x56108cabc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:31.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 4 client.26467 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:14:31.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.822+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56108c9ab0a0 con 0x56108caba1d0 2023-11-17T00:14:31.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.824+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108caba1d0 0x56108cabc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:31.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.824+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108caba1d0 0x56108cabc5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829190dd3b37485b server_cookie=306555ad58234198 in_seq=0 out_seq=0 2023-11-17T00:14:31.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:31.824+0000 7fcffeffd700 10 client.26467 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:32.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:32 smithi006 ceph-mon[97840]: pgmap v172: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 909 op/s 2023-11-17T00:14:33.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:32 smithi087 ceph-mon[131182]: pgmap v172: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 909 op/s 2023-11-17T00:14:33.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:32 smithi186 ceph-mon[131487]: pgmap v172: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 909 op/s 2023-11-17T00:14:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:34 smithi006 ceph-mon[97840]: pgmap v173: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 675 op/s 2023-11-17T00:14:35.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:34 smithi087 ceph-mon[131182]: pgmap v173: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 675 op/s 2023-11-17T00:14:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:34 smithi186 ceph-mon[131487]: pgmap v173: 129 pgs: 129 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 675 op/s 2023-11-17T00:14:36.978 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:36 smithi006 ceph-mon[97840]: pgmap v174: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 1.14k op/s 2023-11-17T00:14:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:36 smithi087 ceph-mon[131182]: pgmap v174: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 1.14k op/s 2023-11-17T00:14:37.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:36 smithi186 ceph-mon[131487]: pgmap v174: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 1.14k op/s 2023-11-17T00:14:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d617fa700 1 -- 172.21.15.6:0/3851473599 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55974e68c600 con 0x55974e7d9d50 2023-11-17T00:14:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d617fa700 10 client.26489 handle_command_reply: tid=0 2023-11-17T00:14:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 client.26489 shutdown 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 msgr2=0x55974e77bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d74033010 tx=0x7f0d740092d0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 shutdown_connections 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55974e772b90 0x55974e779780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55974e5c86b0 0x55974e77bc30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55974e7d9d50 0x55974e7dc130 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 --2- 172.21.15.6:0/3851473599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55974e772720 0x55974e779040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.580+0000 7fcffeffd700 1 -- 172.21.15.6:0/974083633 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56108c9ab0a0 con 0x56108caba1d0 2023-11-17T00:14:38.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.580+0000 7fcffeffd700 10 client.26467 handle_command_reply: tid=0 2023-11-17T00:14:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d617fa700 0 client.26489 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 >> 172.21.15.6:0/3851473599 conn(0x55974e761f50 msgr2=0x55974e7752c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.580+0000 7fd0398f2b80 1 client.26467 shutdown 2023-11-17T00:14:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 shutdown_connections 2023-11-17T00:14:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.580+0000 7f0d937efb80 1 -- 172.21.15.6:0/3851473599 wait complete. 2023-11-17T00:14:38.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 msgr2=0x56108caa1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd020105e80 tx=0x7fd01800bf60 comp rx=0 tx=0).stop 2023-11-17T00:14:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.581+0000 7f0d937efb80 20 client.26489 trim_cache size 0 max 16384 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 shutdown_connections 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caa1e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca97b60 0x56108caa1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108caba1d0 0x56108cabc5e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/974083633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca98470 0x56108caa18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fcffeffd700 0 client.26467 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:14:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:14:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:14:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:14:38.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.581+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 >> 172.21.15.6:0/974083633 conn(0x56108ca8d360 msgr2=0x56108ca9dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.581+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 msgr2=0x7f0d640a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.581+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 0x7f0d640a38c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009b30 tx=0x7f0d74009260 comp rx=0 tx=0).stop 2023-11-17T00:14:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.581+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 msgr2=0x7f0d781a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.581+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d68005910 tx=0x7f0d680047c0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.582+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 shutdown_connections 2023-11-17T00:14:38.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.582+0000 7fd0398f2b80 1 -- 172.21.15.6:0/974083633 wait complete. 2023-11-17T00:14:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 shutdown_connections 2023-11-17T00:14:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f0d640a1410 0x7f0d640a38c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d78072e90 0x7f0d781a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d78111240 0x7f0d7819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 --2- 172.21.15.6:0/1752554399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d780737a0 0x7f0d781a2cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 >> 172.21.15.6:0/1752554399 conn(0x7f0d780fbd10 msgr2=0x7f0d780fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.582+0000 7fd0398f2b80 20 client.26467 trim_cache size 0 max 16384 2023-11-17T00:14:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 shutdown_connections 2023-11-17T00:14:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:38.582+0000 7f0d7fdff700 1 -- 172.21.15.6:0/1752554399 wait complete. 2023-11-17T00:14:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.603+0000 7fd0398f2b80 1 Processor -- start 2023-11-17T00:14:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.603+0000 7fd0398f2b80 1 -- start start 2023-11-17T00:14:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca97b60 0x56108ca9a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108cab11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108ca98bc0 con 0x56108ca9c8b0 2023-11-17T00:14:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108cab6070 con 0x56108ca98470 2023-11-17T00:14:38.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108ca9f940 con 0x56108ca97b60 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45294/0 (socket says 172.21.15.6:45294) 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd01ffff700 1 -- 172.21.15.6:0/1069410624 learned_addr learned my addr 172.21.15.6:0/1069410624 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.604+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca97b60 0x56108ca9a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 -- 172.21.15.6:0/1069410624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca97b60 msgr2=0x56108ca9a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca97b60 0x56108ca9a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 -- 172.21.15.6:0/1069410624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 msgr2=0x56108cab11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108cab11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 -- 172.21.15.6:0/1069410624 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108ca9cf50 con 0x56108ca9c8b0 2023-11-17T00:14:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.605+0000 7fd01ffff700 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd018002230 tx=0x7fd01800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553b70a18dd29135 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.606+0000 7fd0067fc700 1 -- 172.21.15.6:0/1069410624 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800e040 con 0x56108ca9c8b0 2023-11-17T00:14:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.606+0000 7fd0067fc700 1 -- 172.21.15.6:0/1069410624 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd018020070 con 0x56108ca9c8b0 2023-11-17T00:14:38.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.606+0000 7fd0067fc700 1 -- 172.21.15.6:0/1069410624 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018005420 con 0x56108ca9c8b0 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- 172.21.15.6:0/1069410624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 msgr2=0x56108ca9aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd018002230 tx=0x7fd01800b700 comp rx=0 tx=0).stop 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- 172.21.15.6:0/1069410624 shutdown_connections 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca9c8b0 0x56108ca9aa70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca97b60 0x56108ca9a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/1069410624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108cab11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- 172.21.15.6:0/1069410624 >> 172.21.15.6:0/1069410624 conn(0x56108ca8d360 msgr2=0x56108cab5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- 172.21.15.6:0/1069410624 shutdown_connections 2023-11-17T00:14:38.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- 172.21.15.6:0/1069410624 wait complete. 2023-11-17T00:14:38.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.607+0000 7fd0398f2b80 1 -- start start 2023-11-17T00:14:38.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.608+0000 7fd0398f2b80 1 Processor -- start 2023-11-17T00:14:38.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.608+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108caaf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108cb0d240 con 0x56108ca97b60 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108cb0d3b0 con 0x56108ca98470 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108cb0d570 con 0x56108ca9c8b0 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd0398f2b80 10 client.0 _ll_register_callbacks cb 0x7fd01800bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108caaf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108caaf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45310/0 (socket says 172.21.15.6:45310) 2023-11-17T00:14:38.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01f7fe700 1 -- 172.21.15.6:0/704986511 learned_addr learned my addr 172.21.15.6:0/704986511 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:38.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:43964/0 (socket says 172.21.15.6:43964) 2023-11-17T00:14:38.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01ffff700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 -- 172.21.15.6:0/704986511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 msgr2=0x56108caafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:38.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 -- 172.21.15.6:0/704986511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 msgr2=0x56108caaf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:38.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108caaf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.609+0000 7fd01effd700 1 -- 172.21.15.6:0/704986511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56108c9af190 con 0x56108ca98470 2023-11-17T00:14:38.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fd01effd700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd01401c040 tx=0x7fd014005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442f8c4b0a8a3b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:38.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fcffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:14:38.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014012dc0 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd014012910 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd014005630 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd014003680 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fd0398f2b80 10 client.26498 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.610+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56108c816de0 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd014012910 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd0140327f0 con 0x56108ca98470 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 10 client.26498 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 20 client.26498 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108cb0ec00 0x56108cb10fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 4 client.26498 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:14:38.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56108c9b7660 con 0x56108cb0ec00 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.611+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108cb0ec00 0x56108cb10fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.612+0000 7fd01f7fe700 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108cb0ec00 0x56108cb10fe0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25117684a6ea5ee server_cookie=6fc5d1650956f117 in_seq=0 out_seq=0 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.612+0000 7fcffeffd700 10 client.26498 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fcffeffd700 1 -- 172.21.15.6:0/704986511 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x56108c9b7660 con 0x56108cb0ec00 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fcffeffd700 10 client.26498 handle_command_reply: tid=0 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 client.26498 shutdown 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 msgr2=0x56108caaf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd01401c040 tx=0x7fd014005880 comp rx=0 tx=0).stop 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 shutdown_connections 2023-11-17T00:14:38.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56108cb0ec00 0x56108cb10fe0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56108ca97b60 0x56108caaf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56108ca9c8b0 0x56108caafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 --2- 172.21.15.6:0/704986511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56108ca98470 0x56108caaf6e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fcffeffd700 0 client.26498 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.616+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 >> 172.21.15.6:0/704986511 conn(0x56108ca8d360 msgr2=0x56108ca9aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 shutdown_connections 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd0398f2b80 1 -- 172.21.15.6:0/704986511 wait complete. 2023-11-17T00:14:38.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd0398f2b80 20 client.26498 trim_cache size 0 max 16384 2023-11-17T00:14:38.622 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:14:38.622 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (4205 inodes in the stack)", 2023-11-17T00:14:38.622 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:14:38.622 INFO:teuthology.orchestra.run.smithi006.stdout: "9ebf54b4-47aa-46ec-acbc-7e7519866a60": { 2023-11-17T00:14:38.623 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:14:38.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:14:38.623 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:14:38.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:14:38.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:14:38.624 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:14:38.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 msgr2=0x7fd0080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 0x7fd0080a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd01400a010 tx=0x7fd014012f50 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 msgr2=0x7fd020198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.617+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd01000c920 tx=0x7fd01000cce0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 shutdown_connections 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd0080a1210 0x7fd0080a36c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd020105250 0x7fd02019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020067dc0 0x7fd020198640 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 --2- 172.21.15.6:0/1882546371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd020068770 0x7fd020198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 >> 172.21.15.6:0/1882546371 conn(0x7fd0200763a0 msgr2=0x7fd02010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 shutdown_connections 2023-11-17T00:14:38.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:14:38.618+0000 7fd025fff700 1 -- 172.21.15.6:0/1882546371 wait complete. 2023-11-17T00:14:38.642 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebf54b4-47aa-46ec-acbc-7e7519866a60 - {'path': '/', 'tag': '9ebf54b4-47aa-46ec-acbc-7e7519866a60', 'options': 'recursive,force'} 2023-11-17T00:14:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:38 smithi006 ceph-mon[97840]: pgmap v175: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:38 smithi087 ceph-mon[131182]: pgmap v175: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:39.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:38 smithi186 ceph-mon[131487]: pgmap v175: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:40.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:40 smithi006 ceph-mon[97840]: pgmap v176: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:41.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:40 smithi087 ceph-mon[131182]: pgmap v176: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:40 smithi186 ceph-mon[131487]: pgmap v176: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 746 op/s 2023-11-17T00:14:42.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:42 smithi006 ceph-mon[97840]: pgmap v177: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.14k op/s 2023-11-17T00:14:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:42 smithi087 ceph-mon[131182]: pgmap v177: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.14k op/s 2023-11-17T00:14:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:42 smithi186 ceph-mon[131487]: pgmap v177: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 1.14k op/s 2023-11-17T00:14:44.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:44 smithi006 ceph-mon[97840]: pgmap v178: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 860 op/s 2023-11-17T00:14:45.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:44 smithi087 ceph-mon[131182]: pgmap v178: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 860 op/s 2023-11-17T00:14:45.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:44 smithi186 ceph-mon[131487]: pgmap v178: 129 pgs: 129 active+clean; 19 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 860 op/s 2023-11-17T00:14:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:46 smithi006 ceph-mon[97840]: pgmap v179: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.30k op/s 2023-11-17T00:14:47.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:46 smithi087 ceph-mon[131182]: pgmap v179: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.30k op/s 2023-11-17T00:14:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:46 smithi186 ceph-mon[131487]: pgmap v179: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.30k op/s 2023-11-17T00:14:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:48 smithi087 ceph-mon[131182]: pgmap v180: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:48 smithi186 ceph-mon[131487]: pgmap v180: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:49.222 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:48 smithi006 ceph-mon[97840]: pgmap v180: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:51.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:50 smithi087 ceph-mon[131182]: pgmap v181: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:50 smithi186 ceph-mon[131487]: pgmap v181: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:50 smithi006 ceph-mon[97840]: pgmap v181: 129 pgs: 129 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 841 op/s 2023-11-17T00:14:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/1711051230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c980ffb40 msgr2=0x7f9c98104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/1711051230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c980ffb40 0x7f9c98104010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009920 tx=0x7f9c8c009c30 comp rx=0 tx=0).stop 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/1711051230 shutdown_connections 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/1711051230 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c98104630 0x7f9c98104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/1711051230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c980ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/1711051230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c980ffb40 0x7f9c98104010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.747+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/1711051230 >> 172.21.15.6:0/1711051230 conn(0x7f9c980763a0 msgr2=0x7f9c980767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.748+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/1711051230 shutdown_connections 2023-11-17T00:14:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.748+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/1711051230 wait complete. 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 Processor -- start 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 -- start start 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c980ffb40 0x7f9c981a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 0x7f9c9819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819cef0 con 0x7f9c980ffb40 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819d060 con 0x7f9c980ff230 2023-11-17T00:14:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c9d5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819d220 con 0x7f9c98104630 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.749+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c980ffb40 0x7f9c981a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:39228/0 (socket says 172.21.15.6:39228) 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c96ffd700 1 -- 172.21.15.6:0/816698085 learned_addr learned my addr 172.21.15.6:0/816698085 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 0x7f9c9819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 -- 172.21.15.6:0/816698085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 msgr2=0x7f9c981a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 -- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c980ffb40 msgr2=0x7f9c981a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c980ffb40 0x7f9c981a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 -- 172.21.15.6:0/816698085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880095f0 con 0x7f9c98104630 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 0x7f9c9819c880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000efc0 tx=0x7f9c9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c7ffff700 1 -- 172.21.15.6:0/816698085 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90018070 con 0x7f9c98104630 2023-11-17T00:14:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.750+0000 7f9c7ffff700 1 -- 172.21.15.6:0/816698085 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9000f040 con 0x7f9c98104630 2023-11-17T00:14:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.751+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c009520 con 0x7f9c98104630 2023-11-17T00:14:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.751+0000 7f9c7ffff700 1 -- 172.21.15.6:0/816698085 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000cd40 con 0x7f9c98104630 2023-11-17T00:14:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.751+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c981168d0 con 0x7f9c98104630 2023-11-17T00:14:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.755+0000 7f9c7ffff700 1 -- 172.21.15.6:0/816698085 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9c90003a00 con 0x7f9c98104630 2023-11-17T00:14:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 Processor -- start 2023-11-17T00:14:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 -- start start 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62a30000 con 0x55ca62b30720 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62a82840 con 0x55ca62b30b90 2023-11-17T00:14:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62a7bcd0 con 0x55ca629866b0 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.757+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33046/0 (socket says 172.21.15.6:33046) 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 -- 172.21.15.6:0/2880354064 learned_addr learned my addr 172.21.15.6:0/2880354064 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c7ffff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 0x7f9c800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c967fc700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c967fc700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c7ffff700 1 -- 172.21.15.6:0/816698085 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c900ccb40 con 0x7f9c98104630 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 0x7f9c800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 -- 172.21.15.6:0/2880354064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 msgr2=0x55ca62b301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 -- 172.21.15.6:0/2880354064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 msgr2=0x55ca62b2ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c96ffd700 1 -- 172.21.15.6:0/2880354064 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca628fd5d0 con 0x55ca62b30720 2023-11-17T00:14:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.758+0000 7f9c967fc700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.759+0000 7f9c977fe700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 0x7f9c800a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c6f0 tx=0x7f9c900092e0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.759+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9c880118d0 tx=0x7f9c88011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c269a97c4458b00 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.759+0000 7f9c7d7fa700 1 -- 172.21.15.6:0/2880354064 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88011250 con 0x55ca62b30720 2023-11-17T00:14:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.759+0000 7f9c7d7fa700 1 -- 172.21.15.6:0/2880354064 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c88004cf0 con 0x55ca62b30720 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.759+0000 7f9c7d7fa700 1 -- 172.21.15.6:0/2880354064 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800f8a0 con 0x55ca62b30720 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- 172.21.15.6:0/2880354064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 msgr2=0x55ca62b2a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9c880118d0 tx=0x7f9c88011be0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- 172.21.15.6:0/2880354064 shutdown_connections 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/2880354064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b2a560 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- 172.21.15.6:0/2880354064 >> 172.21.15.6:0/2880354064 conn(0x55ca62b1ff50 msgr2=0x55ca62b25270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- 172.21.15.6:0/2880354064 shutdown_connections 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- 172.21.15.6:0/2880354064 wait complete. 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.760+0000 7f9cb1018b80 1 -- start start 2023-11-17T00:14:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 Processor -- start 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b34330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62a1d880 con 0x55ca62b30720 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62a1d520 con 0x55ca62b30b90 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca628ff8c0 con 0x55ca629866b0 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9cb1018b80 10 client.0 _ll_register_callbacks cb 0x55ca62a3b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.761+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33056/0 (socket says 172.21.15.6:33056) 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 -- 172.21.15.6:0/591107991 learned_addr learned my addr 172.21.15.6:0/591107991 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c967fc700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b34330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 -- 172.21.15.6:0/591107991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 msgr2=0x55ca62b36a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b36a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 -- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 msgr2=0x55ca62b34330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b34330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 -- 172.21.15.6:0/591107991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c88011500 con 0x55ca62b30720 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9c880051b0 tx=0x7f9c88005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181504b4b77c55ad server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.762+0000 7f9c7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880113a0 con 0x55ca62b30720 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca62b1d710 con 0x55ca62b30720 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c88004950 con 0x55ca62b30720 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c88004b10 con 0x55ca62b30720 2023-11-17T00:14:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9cb1018b80 10 client.16395 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ca62a42130 con 0x55ca62b30720 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca62b1d710 con 0x55ca62b30720 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.763+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c88021040 con 0x55ca62b30720 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 20 client.16395 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62b43f30 0x55ca62b46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 4 client.16395 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca62a3e040 con 0x55ca62b43f30 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9c967fc700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62b43f30 0x55ca62b46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9c967fc700 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62b43f30 0x55ca62b46340 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a58eb469e1bc164 server_cookie=bcd14b84ae27b251 in_seq=0 out_seq=0 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.764+0000 7f9c7aff5700 10 client.16395 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.765+0000 7f9c7aff5700 1 -- 172.21.15.6:0/591107991 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca62a3e040 con 0x55ca62b43f30 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.765+0000 7f9c7aff5700 10 client.16395 handle_command_reply: tid=0 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 client.16395 shutdown 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 msgr2=0x55ca62b33cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9c880051b0 tx=0x7f9c88005790 comp rx=0 tx=0).stop 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 shutdown_connections 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b34330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b36a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62b43f30 0x55ca62b46340 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/591107991 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b33cd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9c7aff5700 0 client.16395 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 >> 172.21.15.6:0/591107991 conn(0x55ca62b1ff50 msgr2=0x55ca62b247e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 shutdown_connections 2023-11-17T00:14:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 1 -- 172.21.15.6:0/591107991 wait complete. 2023-11-17T00:14:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.766+0000 7f9cb1018b80 20 client.16395 trim_cache size 0 max 16384 2023-11-17T00:14:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.794+0000 7f9cb1018b80 1 Processor -- start 2023-11-17T00:14:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 -- start start 2023-11-17T00:14:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 0x55ca62b36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b2b6a0 con 0x55ca629866b0 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b2e7f0 con 0x55ca62b30b90 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.795+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b26000 con 0x55ca62b30720 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.796+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.796+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 0x55ca62b36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.796+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33060/0 (socket says 172.21.15.6:33060) 2023-11-17T00:14:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.796+0000 7f9c967fc700 1 -- 172.21.15.6:0/3559527103 learned_addr learned my addr 172.21.15.6:0/3559527103 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.796+0000 7f9c967fc700 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c967fc700 1 -- 172.21.15.6:0/3559527103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 msgr2=0x55ca62b36680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c967fc700 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 0x55ca62b36680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c967fc700 1 -- 172.21.15.6:0/3559527103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 msgr2=0x55ca62b2de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c967fc700 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c967fc700 1 -- 172.21.15.6:0/3559527103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca62a41e80 con 0x55ca629866b0 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.797+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 0x55ca62b36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.798+0000 7f9c967fc700 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005850 tx=0x7f9c8c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c008bddc7bb1cb0f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.799+0000 7f9c7dffb700 1 -- 172.21.15.6:0/3559527103 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c01c040 con 0x55ca629866b0 2023-11-17T00:14:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.799+0000 7f9c7dffb700 1 -- 172.21.15.6:0/3559527103 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c8c033070 con 0x55ca629866b0 2023-11-17T00:14:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.799+0000 7f9c7dffb700 1 -- 172.21.15.6:0/3559527103 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00b560 con 0x55ca629866b0 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.800+0000 7f9cb1018b80 1 -- 172.21.15.6:0/3559527103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 msgr2=0x55ca62b24f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005850 tx=0x7f9c8c009c30 comp rx=0 tx=0).stop 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 -- 172.21.15.6:0/3559527103 shutdown_connections 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca629866b0 0x55ca62b24f60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b2de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/3559527103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca62b30720 0x55ca62b36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.801+0000 7f9cb1018b80 1 -- 172.21.15.6:0/3559527103 >> 172.21.15.6:0/3559527103 conn(0x55ca62b1ff50 msgr2=0x55ca62b2a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:14:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.802+0000 7f9cb1018b80 1 -- 172.21.15.6:0/3559527103 shutdown_connections 2023-11-17T00:14:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.802+0000 7f9cb1018b80 1 -- 172.21.15.6:0/3559527103 wait complete. 2023-11-17T00:14:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.805+0000 7f9cb1018b80 1 -- start start 2023-11-17T00:14:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.805+0000 7f9cb1018b80 1 Processor -- start 2023-11-17T00:14:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b3a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b396a0 con 0x55ca62b30720 2023-11-17T00:14:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b39810 con 0x55ca62b30b90 2023-11-17T00:14:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca62b399d0 con 0x55ca629866b0 2023-11-17T00:14:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.807+0000 7f9cb1018b80 10 client.0 _ll_register_callbacks cb 0x7f9c900097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:14:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.808+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b3a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.809+0000 7f9c967fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b3a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:42778/0 (socket says 172.21.15.6:42778) 2023-11-17T00:14:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.809+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.809+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33062/0 (socket says 172.21.15.6:33062) 2023-11-17T00:14:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.809+0000 7f9c96ffd700 1 -- 172.21.15.6:0/1644639785 learned_addr learned my addr 172.21.15.6:0/1644639785 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:14:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.810+0000 7f9c967fc700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.811+0000 7f9c967fc700 1 -- 172.21.15.6:0/1644639785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 msgr2=0x55ca62b3a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.811+0000 7f9c967fc700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b3a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.811+0000 7f9c967fc700 1 -- 172.21.15.6:0/1644639785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 msgr2=0x55ca62b3af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:14:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.812+0000 7f9c967fc700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:14:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.815+0000 7f9c967fc700 1 -- 172.21.15.6:0/1644639785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca62a42130 con 0x55ca62b30b90 2023-11-17T00:14:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.815+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.816+0000 7f9c967fc700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b39080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c010 tx=0x7f9c8c00f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322bb7f5dfc6c5df server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.816+0000 7f9c7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.816+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c01c040 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.816+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c8c005240 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.816+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c88004cf0 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.818+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c8c005400 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.819+0000 7f9cb1018b80 10 client.26516 fetch_fsmap learned FSMap version 22 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.819+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f9c98113f60 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c88004cf0 con 0x55ca62b30b90 2023-11-17T00:14:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c8c015020 con 0x55ca62b30b90 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 20 client.26516 populate_metadata read hostname 'smithi006' 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62ba9960 0x55ca62babd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 4 client.26516 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.820+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca62a4a600 con 0x55ca62ba9960 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.821+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62ba9960 0x55ca62babd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.822+0000 7f9c96ffd700 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62ba9960 0x55ca62babd40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359bdbe240933143 server_cookie=7d18d181dc63de24 in_seq=0 out_seq=0 2023-11-17T00:14:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:51.822+0000 7f9c7aff5700 10 client.26516 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:14:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:52 smithi087 ceph-mon[131182]: pgmap v182: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 1.04k op/s 2023-11-17T00:14:53.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:52 smithi186 ceph-mon[131487]: pgmap v182: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 1.04k op/s 2023-11-17T00:14:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:52 smithi006 ceph-mon[97840]: pgmap v182: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 1.04k op/s 2023-11-17T00:14:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:54 smithi087 ceph-mon[131182]: pgmap v183: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 644 op/s 2023-11-17T00:14:55.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:54 smithi186 ceph-mon[131487]: pgmap v183: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 644 op/s 2023-11-17T00:14:55.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:54 smithi006 ceph-mon[97840]: pgmap v183: 129 pgs: 129 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 644 op/s 2023-11-17T00:14:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9c7aff5700 1 -- 172.21.15.6:0/1644639785 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca62a4a600 con 0x55ca62ba9960 2023-11-17T00:15:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9c7aff5700 10 client.26516 handle_command_reply: tid=0 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 client.26516 shutdown 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 msgr2=0x55ca62b39080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b39080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c010 tx=0x7f9c8c00f460 comp rx=0 tx=0).stop 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 shutdown_connections 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62b30b90 0x55ca62b39080 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ca629866b0 0x55ca62b3a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ca62ba9960 0x55ca62babd40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 --2- 172.21.15.6:0/1644639785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ca62b30720 0x55ca62b3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9c7aff5700 0 client.26516 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.032+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 >> 172.21.15.6:0/1644639785 conn(0x55ca62b1ff50 msgr2=0x55ca62b2de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.033+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 shutdown_connections 2023-11-17T00:15:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.033+0000 7f9cb1018b80 1 -- 172.21.15.6:0/1644639785 wait complete. 2023-11-17T00:15:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.033+0000 7f9cb1018b80 20 client.26516 trim_cache size 0 max 16384 2023-11-17T00:15:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:15:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:15:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:15:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:15:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.033+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 msgr2=0x7f9c800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.033+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 0x7f9c800a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c6f0 tx=0x7f9c900092e0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 msgr2=0x7f9c9819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 0x7f9c9819c880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000efc0 tx=0x7f9c9000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 shutdown_connections 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c800a1340 0x7f9c800a37f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c98104630 0x7f9c9819c880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c980ff230 0x7f9c981a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 --2- 172.21.15.6:0/816698085 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c980ffb40 0x7f9c981a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 >> 172.21.15.6:0/816698085 conn(0x7f9c980763a0 msgr2=0x7f9c981020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 shutdown_connections 2023-11-17T00:15:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:14:57.034+0000 7f9c9d5ff700 1 -- 172.21.15.6:0/816698085 wait complete. 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:56 smithi006 ceph-mon[97840]: pgmap v184: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 1.07k op/s 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:14:58 smithi006 ceph-mon[97840]: pgmap v185: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:00 smithi006 ceph-mon[97840]: pgmap v186: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:02 smithi006 ceph-mon[97840]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 884 op/s 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:04 smithi006 ceph-mon[97840]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 684 op/s 2023-11-17T00:15:08.818 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:06 smithi006 ceph-mon[97840]: pgmap v189: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2023-11-17T00:15:08.845 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:07 smithi006 ceph-mon[97840]: osdmap e94: 12 total, 12 up, 12 in 2023-11-17T00:15:08.846 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:15:08.850 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:56 smithi186 ceph-mon[131487]: pgmap v184: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 1.07k op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:14:58 smithi186 ceph-mon[131487]: pgmap v185: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:00 smithi186 ceph-mon[131487]: pgmap v186: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:02 smithi186 ceph-mon[131487]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 884 op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:04 smithi186 ceph-mon[131487]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 684 op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:06 smithi186 ceph-mon[131487]: pgmap v189: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:07 smithi186 ceph-mon[131487]: osdmap e94: 12 total, 12 up, 12 in 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:56 smithi087 ceph-mon[131182]: pgmap v184: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 1.07k op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:14:58 smithi087 ceph-mon[131182]: pgmap v185: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.881 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:00 smithi087 ceph-mon[131182]: pgmap v186: 129 pgs: 129 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2023-11-17T00:15:08.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:02 smithi087 ceph-mon[131182]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 884 op/s 2023-11-17T00:15:08.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:04 smithi087 ceph-mon[131182]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 684 op/s 2023-11-17T00:15:08.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:06 smithi087 ceph-mon[131182]: pgmap v189: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2023-11-17T00:15:08.882 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:07 smithi087 ceph-mon[131182]: osdmap e94: 12 total, 12 up, 12 in 2023-11-17T00:15:09.108 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:08 smithi087 ceph-mon[131182]: pgmap v191: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:09.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:08 smithi186 ceph-mon[131487]: pgmap v191: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:09.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.218+0000 7fa1b35ff700 1 -- 172.21.15.6:0/3810621205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 msgr2=0x7fa1ac10f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.218+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/3810621205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac10f230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009f90 tx=0x7fa1a400b330 comp rx=0 tx=0).stop 2023-11-17T00:15:09.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 -- 172.21.15.6:0/3810621205 shutdown_connections 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/3810621205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac10f230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/3810621205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac0746a0 0x7fa1ac072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/3810621205 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac074b00 0x7fa1ac073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 -- 172.21.15.6:0/3810621205 >> 172.21.15.6:0/3810621205 conn(0x7fa1ac0774f0 msgr2=0x7fa1ac0778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 -- 172.21.15.6:0/3810621205 shutdown_connections 2023-11-17T00:15:09.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.219+0000 7fa1b35ff700 1 -- 172.21.15.6:0/3810621205 wait complete. 2023-11-17T00:15:09.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.220+0000 7fa1b35ff700 1 Processor -- start 2023-11-17T00:15:09.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 -- start start 2023-11-17T00:15:09.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac0746a0 0x7fa1ac1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac074b00 0x7fa1ac198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac19c5a0 con 0x7fa1ac0746a0 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac19c710 con 0x7fa1ac10ce50 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b35ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac19c8d0 con 0x7fa1ac074b00 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.221+0000 7fa1b139b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac0746a0 0x7fa1ac1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46134/0 (socket says 172.21.15.6:46134) 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/167532737 learned_addr learned my addr 172.21.15.6:0/167532737 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:09.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac074b00 0x7fa1ac198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:08 smithi006 ceph-mon[97840]: pgmap v191: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/167532737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac074b00 msgr2=0x7fa1ac198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac074b00 0x7fa1ac198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac0746a0 msgr2=0x7fa1ac1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac0746a0 0x7fa1ac1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/167532737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4009c40 con 0x7fa1ac10ce50 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.222+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40094b0 tx=0x7fa1a40094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.223+0000 7fa1a27fc700 1 -- 172.21.15.6:0/167532737 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400e040 con 0x7fa1ac10ce50 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.223+0000 7fa1a27fc700 1 -- 172.21.15.6:0/167532737 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a4004e70 con 0x7fa1ac10ce50 2023-11-17T00:15:09.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.223+0000 7fa1a27fc700 1 -- 172.21.15.6:0/167532737 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005040 con 0x7fa1ac10ce50 2023-11-17T00:15:09.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.223+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac112520 con 0x7fa1ac10ce50 2023-11-17T00:15:09.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.223+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac112390 con 0x7fa1ac10ce50 2023-11-17T00:15:09.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.227+0000 7fa1a27fc700 1 -- 172.21.15.6:0/167532737 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa1a401d470 con 0x7fa1ac10ce50 2023-11-17T00:15:09.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.228+0000 7fa1c6f2bb80 1 Processor -- start 2023-11-17T00:15:09.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.228+0000 7fa1c6f2bb80 1 -- start start 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2626370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1a27fc700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 0x7fa1980a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2626d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2527060 con 0x558dd26268b0 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd25798a0 con 0x558dd2621b60 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2572d30 con 0x558dd2622470 2023-11-17T00:15:09.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1a27fc700 1 -- 172.21.15.6:0/167532737 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa1a40cdd80 con 0x7fa1ac10ce50 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1b0b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1b0b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46150/0 (socket says 172.21.15.6:46150) 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1b0b9a700 1 -- 172.21.15.6:0/157678112 learned_addr learned my addr 172.21.15.6:0/157678112 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.229+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2626370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2626d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b139b700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 0x7fa1980a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 -- 172.21.15.6:0/157678112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 msgr2=0x558dd2626370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2626370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 -- 172.21.15.6:0/157678112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 msgr2=0x558dd2626d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2626d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 -- 172.21.15.6:0/157678112 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd23f4630 con 0x558dd2621b60 2023-11-17T00:15:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.230+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0137e0 tx=0x7fa19c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ed2a8633548a4e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.231+0000 7fa1977fe700 1 -- 172.21.15.6:0/157678112 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c013e00 con 0x558dd2621b60 2023-11-17T00:15:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.231+0000 7fa1b139b700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 0x7fa1980a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8009310 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.231+0000 7fa1977fe700 1 -- 172.21.15.6:0/157678112 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa19c0044f0 con 0x558dd2621b60 2023-11-17T00:15:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.231+0000 7fa1977fe700 1 -- 172.21.15.6:0/157678112 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c005020 con 0x558dd2621b60 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/157678112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 msgr2=0x558dd2621f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0137e0 tx=0x7fa19c009750 comp rx=0 tx=0).stop 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/157678112 shutdown_connections 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2626d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2621f30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/157678112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2626370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/157678112 >> 172.21.15.6:0/157678112 conn(0x558dd2617360 msgr2=0x558dd261cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/157678112 shutdown_connections 2023-11-17T00:15:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.232+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/157678112 wait complete. 2023-11-17T00:15:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.233+0000 7fa1c6f2bb80 1 -- start start 2023-11-17T00:15:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.233+0000 7fa1c6f2bb80 1 Processor -- start 2023-11-17T00:15:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.233+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 0x558dd262e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.233+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd262b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.233+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd25148e0 con 0x558dd2622470 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2514580 con 0x558dd26268b0 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd23f6920 con 0x558dd2621b60 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1c6f2bb80 10 client.0 _ll_register_callbacks cb 0x558dd2532780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46158/0 (socket says 172.21.15.6:46158) 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/187330600 learned_addr learned my addr 172.21.15.6:0/187330600 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/187330600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 msgr2=0x558dd262e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b139b700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 0x558dd262e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 0x558dd262e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/187330600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 msgr2=0x558dd262b290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.234+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd262b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd262b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1b1b9c700 1 -- 172.21.15.6:0/187330600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c01b040 con 0x558dd26268b0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1b139b700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 0x558dd262e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd262b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40060b0 tx=0x7fa1a409a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732d3574ed6e1135 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa194ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a409aa80 con 0x558dd26268b0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dd2614b20 con 0x558dd26268b0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa1a409eab0 con 0x558dd26268b0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a40a2e40 con 0x558dd26268b0 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1c6f2bb80 10 client.26528 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.235+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558dd2539190 con 0x558dd26268b0 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558dd2614b20 con 0x558dd26268b0 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1a409adc0 con 0x558dd26268b0 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 20 client.26528 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd263b650 0x558dd263da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 4 client.26528 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dd25350a0 con 0x558dd263b650 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.236+0000 7fa1b139b700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd263b650 0x558dd263da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.237+0000 7fa1b139b700 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd263b650 0x558dd263da60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181b1234d226050 server_cookie=8b0c3ed8f5b58fe6 in_seq=0 out_seq=0 2023-11-17T00:15:09.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.237+0000 7fa194ff9700 10 client.26528 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:09.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.241+0000 7fa194ff9700 1 -- 172.21.15.6:0/187330600 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558dd25350a0 con 0x558dd263b650 2023-11-17T00:15:09.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.241+0000 7fa194ff9700 10 client.26528 handle_command_reply: tid=0 2023-11-17T00:15:09.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.241+0000 7fa1c6f2bb80 1 client.26528 shutdown 2023-11-17T00:15:09.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 msgr2=0x558dd262b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40060b0 tx=0x7fa1a409a670 comp rx=0 tx=0).stop 2023-11-17T00:15:09.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 shutdown_connections 2023-11-17T00:15:09.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd26268b0 0x558dd262b910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2621b60 0x558dd262e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd263b650 0x558dd263da60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/187330600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd262b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 >> 172.21.15.6:0/187330600 conn(0x558dd2617360 msgr2=0x558dd2627d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa194ff9700 0 client.26528 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:09.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 shutdown_connections 2023-11-17T00:15:09.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/187330600 wait complete. 2023-11-17T00:15:09.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.242+0000 7fa1c6f2bb80 20 client.26528 trim_cache size 0 max 16384 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.260+0000 7fa1c6f2bb80 1 Processor -- start 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 -- start start 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd2632940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 0x558dd26248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2622bc0 con 0x558dd2622470 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd262bca0 con 0x558dd2621b60 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2616ee0 con 0x558dd26268b0 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b0b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd2632940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b0b9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd2632940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45886/0 (socket says 172.21.15.6:45886) 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46172/0 (socket says 172.21.15.6:46172) 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b0b9a700 1 -- 172.21.15.6:0/1699764203 learned_addr learned my addr 172.21.15.6:0/1699764203 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.261+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 0x558dd26248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 -- 172.21.15.6:0/1699764203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 msgr2=0x558dd26248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 0x558dd26248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 -- 172.21.15.6:0/1699764203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 msgr2=0x558dd2632940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd2632940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 -- 172.21.15.6:0/1699764203 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd2618b00 con 0x558dd2621b60 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 0x558dd26248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x558dd263ba20 tx=0x7fa1a8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6754121fe3954f7c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa197fff700 1 -- 172.21.15.6:0/1699764203 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8026320 con 0x558dd2621b60 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa197fff700 1 -- 172.21.15.6:0/1699764203 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a8026480 con 0x558dd2621b60 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.262+0000 7fa197fff700 1 -- 172.21.15.6:0/1699764203 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801e680 con 0x558dd2621b60 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1699764203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 msgr2=0x558dd261cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x558dd263ba20 tx=0x7fa1a8014000 comp rx=0 tx=0).stop 2023-11-17T00:15:09.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1699764203 shutdown_connections 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd26268b0 0x558dd26248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd261cbe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1699764203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd2622470 0x558dd2632940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1699764203 >> 172.21.15.6:0/1699764203 conn(0x558dd2617360 msgr2=0x558dd2637a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1699764203 shutdown_connections 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.263+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1699764203 wait complete. 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 -- start start 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 Processor -- start 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2688e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2630930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2631010 con 0x558dd26268b0 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2631180 con 0x558dd2621b60 2023-11-17T00:15:09.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd2631340 con 0x558dd2622470 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.264+0000 7fa1c6f2bb80 10 client.0 _ll_register_callbacks cb 0x7fa1a4096dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46180/0 (socket says 172.21.15.6:46180) 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 -- 172.21.15.6:0/1470863859 learned_addr learned my addr 172.21.15.6:0/1470863859 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2688e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2630930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 -- 172.21.15.6:0/1470863859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 msgr2=0x558dd2688e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2688e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 -- 172.21.15.6:0/1470863859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 msgr2=0x558dd2630930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2630930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 -- 172.21.15.6:0/1470863859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a8017070 con 0x558dd2621b60 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b1b9c700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2630930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:15:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa1b139b700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8004200 tx=0x7fa1a80057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8e787294d4d945 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.265+0000 7fa194ff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8003880 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa1a8023030 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa19c0044f0 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a80039e0 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 10 client.26534 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558dd23a0de0 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa19c0044f0 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1a8035070 con 0x558dd2621b60 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 10 client.26534 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 20 client.26534 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd2690550 0x558dd2692960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 4 client.26534 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.266+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dd2541660 con 0x558dd2690550 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.267+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd2690550 0x558dd2692960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.267+0000 7fa194ff9700 10 client.26534 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.267+0000 7fa1b0b9a700 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd2690550 0x558dd2692960 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589e06d27b2d11d8 server_cookie=604da551920c2916 in_seq=0 out_seq=0 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa194ff9700 1 -- 172.21.15.6:0/1470863859 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x558dd2541660 con 0x558dd2690550 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa194ff9700 10 client.26534 handle_command_reply: tid=0 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 client.26534 shutdown 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 msgr2=0x558dd2688940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8004200 tx=0x7fa1a80057d0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 shutdown_connections 2023-11-17T00:15:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558dd2690550 0x558dd2692960 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558dd26268b0 0x558dd2630930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558dd2621b60 0x558dd2688940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 --2- 172.21.15.6:0/1470863859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558dd2622470 0x558dd2688e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa194ff9700 0 client.26534 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.273+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 >> 172.21.15.6:0/1470863859 conn(0x558dd2617360 msgr2=0x558dd26248f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.274+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 shutdown_connections 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.274+0000 7fa1c6f2bb80 1 -- 172.21.15.6:0/1470863859 wait complete. 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.274+0000 7fa1c6f2bb80 20 client.26534 trim_cache size 0 max 16384 2023-11-17T00:15:09.357 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (2887 inodes in the stack)", 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "9ebf54b4-47aa-46ec-acbc-7e7519866a60": { 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:15:09.358 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:15:09.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.274+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 msgr2=0x7fa1980a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.274+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 0x7fa1980a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005fd0 tx=0x7fa1a8009310 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 msgr2=0x7fa1ac19bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40094b0 tx=0x7fa1a40094e0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 shutdown_connections 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa1980a1340 0x7fa1980a37f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ac0746a0 0x7fa1ac1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ac10ce50 0x7fa1ac19bfa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 --2- 172.21.15.6:0/167532737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1ac074b00 0x7fa1ac198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 >> 172.21.15.6:0/167532737 conn(0x7fa1ac0774f0 msgr2=0x7fa1ac104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 shutdown_connections 2023-11-17T00:15:09.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:09.275+0000 7fa1b35ff700 1 -- 172.21.15.6:0/167532737 wait complete. 2023-11-17T00:15:09.361 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebf54b4-47aa-46ec-acbc-7e7519866a60 - {'path': '/', 'tag': '9ebf54b4-47aa-46ec-acbc-7e7519866a60', 'options': 'recursive,force'} 2023-11-17T00:15:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.209+0000 7fa8fbbff700 1 -- 172.21.15.6:0/1171850560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f40737a0 msgr2=0x7fa8f4110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.209+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/1171850560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f40737a0 0x7fa8f4110d20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009990 tx=0x7fa8e4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 -- 172.21.15.6:0/1171850560 shutdown_connections 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/1171850560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f4072e90 0x7fa8f4073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/1171850560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 0x7fa8f4113640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/1171850560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f40737a0 0x7fa8f4110d20 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009990 tx=0x7fa8e4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 -- 172.21.15.6:0/1171850560 >> 172.21.15.6:0/1171850560 conn(0x7fa8f40fbd10 msgr2=0x7fa8f40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.210+0000 7fa8fbbff700 1 -- 172.21.15.6:0/1171850560 shutdown_connections 2023-11-17T00:15:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.211+0000 7fa8fbbff700 1 -- 172.21.15.6:0/1171850560 wait complete. 2023-11-17T00:15:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.214+0000 7fa8fbbff700 1 Processor -- start 2023-11-17T00:15:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 -- start start 2023-11-17T00:15:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f4072e90 0x7fa8f4071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 0x7fa8f4072550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f406bb20 con 0x7fa8f406afe0 2023-11-17T00:15:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f406bc90 con 0x7fa8f4111260 2023-11-17T00:15:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.216+0000 7fa8fbbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f406be50 con 0x7fa8f4072e90 2023-11-17T00:15:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.217+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.217+0000 7fa8f999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f4072e90 0x7fa8f4071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.217+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45916/0 (socket says 172.21.15.6:45916) 2023-11-17T00:15:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.217+0000 7fa8fa19c700 1 -- 172.21.15.6:0/3685507829 learned_addr learned my addr 172.21.15.6:0/3685507829 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.217+0000 7fa8f919a700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 0x7fa8f4072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 -- 172.21.15.6:0/3685507829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f4072e90 msgr2=0x7fa8f4071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f4072e90 0x7fa8f4071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 -- 172.21.15.6:0/3685507829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 msgr2=0x7fa8f4072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 0x7fa8f4072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 -- 172.21.15.6:0/3685507829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0009520 con 0x7fa8f406afe0 2023-11-17T00:15:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.218+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ef90 tx=0x7fa8ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.219+0000 7fa8eaffd700 1 -- 172.21.15.6:0/3685507829 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec018070 con 0x7fa8f406afe0 2023-11-17T00:15:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.219+0000 7fa8eaffd700 1 -- 172.21.15.6:0/3685507829 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8ec00f040 con 0x7fa8f406afe0 2023-11-17T00:15:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.219+0000 7fa8eaffd700 1 -- 172.21.15.6:0/3685507829 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00cc70 con 0x7fa8f406afe0 2023-11-17T00:15:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.219+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e40095f0 con 0x7fa8f406afe0 2023-11-17T00:15:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.219+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4116950 con 0x7fa8f406afe0 2023-11-17T00:15:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.228+0000 7fa90f572b80 1 Processor -- start 2023-11-17T00:15:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.228+0000 7fa8eaffd700 1 -- 172.21.15.6:0/3685507829 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa8ec003a00 con 0x7fa8f406afe0 2023-11-17T00:15:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.230+0000 7fa8eaffd700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 0x7fa8e00a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.230+0000 7fa8eaffd700 1 -- 172.21.15.6:0/3685507829 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa8ec0ccb20 con 0x7fa8f406afe0 2023-11-17T00:15:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.231+0000 7fa90f572b80 1 -- start start 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.231+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b35cc6b0 0x55d2b37761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.231+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b3776720 0x55d2b3770560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.231+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa90f572b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3676000 con 0x55d2b35cc6b0 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa90f572b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b36c8840 con 0x55d2b3776b90 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa90f572b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b36c1cd0 con 0x55d2b3776720 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46210/0 (socket says 172.21.15.6:46210) 2023-11-17T00:15:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 -- 172.21.15.6:0/680472302 learned_addr learned my addr 172.21.15.6:0/680472302 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8f999b700 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b35cc6b0 0x55d2b37761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 -- 172.21.15.6:0/680472302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b3776720 msgr2=0x55d2b3770560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b3776720 0x55d2b3770560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 -- 172.21.15.6:0/680472302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b35cc6b0 msgr2=0x55d2b37761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b35cc6b0 0x55d2b37761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.232+0000 7fa8fa19c700 1 -- 172.21.15.6:0/680472302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2b35435d0 con 0x55d2b3776b90 2023-11-17T00:15:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.233+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0090a0 tx=0x7fa8ec095c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8922175395ccd0d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.233+0000 7fa8f919a700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 0x7fa8e00a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.233+0000 7fa8dffff700 1 -- 172.21.15.6:0/680472302 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec09d850 con 0x55d2b3776b90 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.234+0000 7fa8dffff700 1 -- 172.21.15.6:0/680472302 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8ec0a1be0 con 0x55d2b3776b90 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.234+0000 7fa8dffff700 1 -- 172.21.15.6:0/680472302 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0b9070 con 0x55d2b3776b90 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.234+0000 7fa90f572b80 1 -- 172.21.15.6:0/680472302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 msgr2=0x55d2b3770b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.234+0000 7fa90f572b80 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0090a0 tx=0x7fa8ec095c90 comp rx=0 tx=0).stop 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 -- 172.21.15.6:0/680472302 shutdown_connections 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b35cc6b0 0x55d2b37761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3770b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 --2- 172.21.15.6:0/680472302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b3776720 0x55d2b3770560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 -- 172.21.15.6:0/680472302 >> 172.21.15.6:0/680472302 conn(0x55d2b3765f50 msgr2=0x55d2b376b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 -- 172.21.15.6:0/680472302 shutdown_connections 2023-11-17T00:15:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa90f572b80 1 -- 172.21.15.6:0/680472302 wait complete. 2023-11-17T00:15:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.235+0000 7fa8f919a700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 0x7fa8e00a38b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006010 tx=0x7fa8e400b6b0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.236+0000 7fa90f572b80 1 -- start start 2023-11-17T00:15:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.236+0000 7fa90f572b80 1 Processor -- start 2023-11-17T00:15:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.237+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.237+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3779d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.238+0000 7fa8f999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.238+0000 7fa8f999b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43594/0 (socket says 172.21.15.6:43594) 2023-11-17T00:15:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.238+0000 7fa8f999b700 1 -- 172.21.15.6:0/2471848288 learned_addr learned my addr 172.21.15.6:0/2471848288 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.238+0000 7fa8f919a700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3779d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.238+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b377a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.239+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3663880 con 0x55d2b3776b90 2023-11-17T00:15:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.239+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3663520 con 0x55d2b3776720 2023-11-17T00:15:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.239+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b35458c0 con 0x55d2b35cc6b0 2023-11-17T00:15:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.239+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b377a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.239+0000 7fa90f572b80 10 client.0 _ll_register_callbacks cb 0x55d2b3681720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.240+0000 7fa8fa19c700 1 -- 172.21.15.6:0/2471848288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 msgr2=0x55d2b377caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.240+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.240+0000 7fa8fa19c700 1 -- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 msgr2=0x55d2b3779d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.240+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3779d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.240+0000 7fa8fa19c700 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4116950 con 0x55d2b3776b90 2023-11-17T00:15:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.241+0000 7fa8f999b700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.241+0000 7fa8f919a700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3779d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:15:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.242+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b377a460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec097e10 tx=0x7fa8ec004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98931f0be36c8584 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.243+0000 7fa8dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:15:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.243+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec097bc0 con 0x55d2b3776b90 2023-11-17T00:15:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.243+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa8ec0ca090 con 0x55d2b3776b90 2023-11-17T00:15:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.243+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2b3763710 con 0x55d2b3776b90 2023-11-17T00:15:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.244+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8ec0ca270 con 0x55d2b3776b90 2023-11-17T00:15:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.244+0000 7fa90f572b80 10 client.16422 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.244+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d2b3688130 con 0x55d2b3776b90 2023-11-17T00:15:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2b3763710 con 0x55d2b3776b90 2023-11-17T00:15:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8ec0a1750 con 0x55d2b3776b90 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 20 client.16422 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b3789b20 0x55d2b378bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 4 client.16422 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:15:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.246+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2b3684040 con 0x55d2b3789b20 2023-11-17T00:15:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.247+0000 7fa8f999b700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b3789b20 0x55d2b378bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.248+0000 7fa8f999b700 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b3789b20 0x55d2b378bf30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7131958397d93edd server_cookie=4b60140ff40e646 in_seq=0 out_seq=0 2023-11-17T00:15:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.248+0000 7fa8dd7fa700 10 client.16422 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/2471848288 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2b3684040 con 0x55d2b3789b20 2023-11-17T00:15:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa8dd7fa700 10 client.16422 handle_command_reply: tid=0 2023-11-17T00:15:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 client.16422 shutdown 2023-11-17T00:15:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 msgr2=0x55d2b377a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b377a460 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec097e10 tx=0x7fa8ec004b80 comp rx=0 tx=0).stop 2023-11-17T00:15:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 shutdown_connections 2023-11-17T00:15:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b3789b20 0x55d2b378bf30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b377a460 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b377caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 --2- 172.21.15.6:0/2471848288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3779d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 >> 172.21.15.6:0/2471848288 conn(0x55d2b3765f50 msgr2=0x55d2b376a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.255+0000 7fa8dd7fa700 0 client.16422 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.256+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 shutdown_connections 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.256+0000 7fa90f572b80 1 -- 172.21.15.6:0/2471848288 wait complete. 2023-11-17T00:15:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.256+0000 7fa90f572b80 20 client.16422 trim_cache size 0 max 16384 2023-11-17T00:15:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 Processor -- start 2023-11-17T00:15:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 -- start start 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3781090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 0x55d2b376af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b376cbc0 con 0x55d2b3776720 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3765b70 con 0x55d2b3776b90 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa90f572b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b37782d0 con 0x55d2b35cc6b0 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa8fa19c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46222/0 (socket says 172.21.15.6:46222) 2023-11-17T00:15:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa8fa19c700 1 -- 172.21.15.6:0/1631706919 learned_addr learned my addr 172.21.15.6:0/1631706919 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa8f999b700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3781090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.275+0000 7fa8f919a700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 0x55d2b376af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 -- 172.21.15.6:0/1631706919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 msgr2=0x55d2b3781090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3781090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 -- 172.21.15.6:0/1631706919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 msgr2=0x55d2b376af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 0x55d2b376af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 -- 172.21.15.6:0/1631706919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2b3687e80 con 0x55d2b3776b90 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8f999b700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3781090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8f919a700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 0x55d2b376af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c6f0 tx=0x7fa8ec004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7381b8db451980ad server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8e8bff700 1 -- 172.21.15.6:0/1631706919 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec099180 con 0x55d2b3776b90 2023-11-17T00:15:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8e8bff700 1 -- 172.21.15.6:0/1631706919 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8ec098780 con 0x55d2b3776b90 2023-11-17T00:15:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.276+0000 7fa8e8bff700 1 -- 172.21.15.6:0/1631706919 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec01baf0 con 0x55d2b3776b90 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 -- 172.21.15.6:0/1631706919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 msgr2=0x55d2b3773ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c6f0 tx=0x7fa8ec004d50 comp rx=0 tx=0).stop 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 -- 172.21.15.6:0/1631706919 shutdown_connections 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776b90 0x55d2b3773ee0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3781090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 --2- 172.21.15.6:0/1631706919 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776720 0x55d2b376af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 -- 172.21.15.6:0/1631706919 >> 172.21.15.6:0/1631706919 conn(0x55d2b3765f50 msgr2=0x55d2b377c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 -- 172.21.15.6:0/1631706919 shutdown_connections 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.277+0000 7fa90f572b80 1 -- 172.21.15.6:0/1631706919 wait complete. 2023-11-17T00:15:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 -- start start 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 Processor -- start 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3780ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b3772bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3773100 con 0x55d2b3776b90 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3773270 con 0x55d2b3776720 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2b3773430 con 0x55d2b35cc6b0 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.278+0000 7fa90f572b80 10 client.0 _ll_register_callbacks cb 0x7fa8ec009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46232/0 (socket says 172.21.15.6:46232) 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 -- 172.21.15.6:0/984432461 learned_addr learned my addr 172.21.15.6:0/984432461 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f999b700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3780ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8fa19c700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b3772bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 -- 172.21.15.6:0/984432461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 msgr2=0x55d2b3780ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3780ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 -- 172.21.15.6:0/984432461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 msgr2=0x55d2b3772bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b3772bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f919a700 1 -- 172.21.15.6:0/984432461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2b3688130 con 0x55d2b3776720 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.279+0000 7fa8f999b700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3780ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8f919a700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009990 tx=0x7fa8e4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7073397677b7fb58 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e402b030 con 0x55d2b3776720 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8ec0a1be0 con 0x55d2b3776720 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa8e400f660 con 0x55d2b3776720 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8e4004e80 con 0x55d2b3776720 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa90f572b80 10 client.26552 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d2b34d8c00 con 0x55d2b3776720 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8ec0a1be0 con 0x55d2b3776720 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.280+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8e4032830 con 0x55d2b3776720 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 20 client.26552 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b37ddea0 0x55d2b37e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 4 client.26552 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2b3690600 con 0x55d2b37ddea0 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa8f999b700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b37ddea0 0x55d2b37e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa8f999b700 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b37ddea0 0x55d2b37e02b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f29ecf376608119 server_cookie=4080d60692c47a24 in_seq=0 out_seq=0 2023-11-17T00:15:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:10.281+0000 7fa8dd7fa700 10 client.26552 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:11.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:10 smithi087 ceph-mon[131182]: pgmap v192: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:11.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:10 smithi186 ceph-mon[131487]: pgmap v192: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:10 smithi006 ceph-mon[97840]: pgmap v192: 129 pgs: 129 active+clean; 29 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 894 op/s 2023-11-17T00:15:13.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:12 smithi087 ceph-mon[131182]: pgmap v193: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:12 smithi186 ceph-mon[131487]: pgmap v193: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:12 smithi006 ceph-mon[97840]: pgmap v193: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:14 smithi087 ceph-mon[131182]: pgmap v194: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:14.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:14 smithi087 ceph-mon[131182]: osdmap e95: 12 total, 12 up, 12 in 2023-11-17T00:15:14.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:14 smithi186 ceph-mon[131487]: pgmap v194: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:14.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:14 smithi186 ceph-mon[131487]: osdmap e95: 12 total, 12 up, 12 in 2023-11-17T00:15:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:14 smithi006 ceph-mon[97840]: pgmap v194: 129 pgs: 129 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 923 op/s 2023-11-17T00:15:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:14 smithi006 ceph-mon[97840]: osdmap e95: 12 total, 12 up, 12 in 2023-11-17T00:15:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:16 smithi087 ceph-mon[131182]: pgmap v196: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.11k op/s 2023-11-17T00:15:16.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:16 smithi186 ceph-mon[131487]: pgmap v196: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.11k op/s 2023-11-17T00:15:16.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:16 smithi006 ceph-mon[97840]: pgmap v196: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.11k op/s 2023-11-17T00:15:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.636+0000 7fa8dd7fa700 1 -- 172.21.15.6:0/984432461 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2b3690600 con 0x55d2b37ddea0 2023-11-17T00:15:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.636+0000 7fa8dd7fa700 10 client.26552 handle_command_reply: tid=0 2023-11-17T00:15:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.636+0000 7fa90f572b80 1 client.26552 shutdown 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.636+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 msgr2=0x55d2b3772680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.636+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009990 tx=0x7fa8e4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 shutdown_connections 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d2b37ddea0 0x55d2b37e02b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d2b3776b90 0x55d2b3772bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d2b35cc6b0 0x55d2b3780ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 --2- 172.21.15.6:0/984432461 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d2b3776720 0x55d2b3772680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 >> 172.21.15.6:0/984432461 conn(0x55d2b3765f50 msgr2=0x55d2b3774e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa8dd7fa700 0 client.26552 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 shutdown_connections 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.637+0000 7fa90f572b80 1 -- 172.21.15.6:0/984432461 wait complete. 2023-11-17T00:15:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.639+0000 7fa90f572b80 20 client.26552 trim_cache size 0 max 16384 2023-11-17T00:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:15:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.643+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 msgr2=0x7fa8e00a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 0x7fa8e00a38b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4006010 tx=0x7fa8e400b6b0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 msgr2=0x7fa8f406b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ef90 tx=0x7fa8ec00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 shutdown_connections 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8e00a1400 0x7fa8e00a38b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8f406afe0 0x7fa8f406b480 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8f4072e90 0x7fa8f4071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 --2- 172.21.15.6:0/3685507829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8f4111260 0x7fa8f4072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.644+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 >> 172.21.15.6:0/3685507829 conn(0x7fa8f40fbd10 msgr2=0x7fa8f40fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.645+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 shutdown_connections 2023-11-17T00:15:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:17.645+0000 7fa8fbbff700 1 -- 172.21.15.6:0/3685507829 wait complete. 2023-11-17T00:15:18.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:18 smithi087 ceph-mon[131182]: pgmap v197: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:18.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:18 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:15:18.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:18 smithi186 ceph-mon[131487]: pgmap v197: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:18.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:18 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:15:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:18 smithi006 ceph-mon[97840]: pgmap v197: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:18 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:15:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: pgmap v198: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:15:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:15:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:15:20.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:15:20.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:15:20.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:15:20.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:20 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: pgmap v198: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:15:20.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:20 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: pgmap v198: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 964 op/s 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:15:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:20 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:15:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:22 smithi087 ceph-mon[131182]: pgmap v199: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:22.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:22 smithi186 ceph-mon[131487]: pgmap v199: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:22 smithi006 ceph-mon[97840]: pgmap v199: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:24.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:24 smithi087 ceph-mon[131182]: pgmap v200: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:24.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:24 smithi186 ceph-mon[131487]: pgmap v200: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:24.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:24 smithi006 ceph-mon[97840]: pgmap v200: 129 pgs: 129 active+clean; 25 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 883 op/s 2023-11-17T00:15:26.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:26 smithi087 ceph-mon[131182]: pgmap v201: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 1.11k op/s 2023-11-17T00:15:26.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:26 smithi186 ceph-mon[131487]: pgmap v201: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 1.11k op/s 2023-11-17T00:15:26.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:26 smithi006 ceph-mon[97840]: pgmap v201: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 1.11k op/s 2023-11-17T00:15:28.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:28 smithi087 ceph-mon[131182]: pgmap v202: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:28.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:28 smithi186 ceph-mon[131487]: pgmap v202: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:28 smithi006 ceph-mon[97840]: pgmap v202: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:30.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:30 smithi087 ceph-mon[131182]: pgmap v203: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:30.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:30 smithi186 ceph-mon[131487]: pgmap v203: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 -- 172.21.15.6:0/84210396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 msgr2=0x7f5f5c11be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 --2- 172.21.15.6:0/84210396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c11be20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5f440098f0 tx=0x7f5f44009c00 comp rx=0 tx=0).stop 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 -- 172.21.15.6:0/84210396 shutdown_connections 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 --2- 172.21.15.6:0/84210396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c11be20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 --2- 172.21.15.6:0/84210396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 --2- 172.21.15.6:0/84210396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107ec0 0x7f5f5c108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 -- 172.21.15.6:0/84210396 >> 172.21.15.6:0/84210396 conn(0x7f5f5c06c490 msgr2=0x7f5f5c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.869+0000 7f5f60dff700 1 -- 172.21.15.6:0/84210396 shutdown_connections 2023-11-17T00:15:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.870+0000 7f5f60dff700 1 -- 172.21.15.6:0/84210396 wait complete. 2023-11-17T00:15:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.871+0000 7f5f60dff700 1 Processor -- start 2023-11-17T00:15:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 -- start start 2023-11-17T00:15:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f5c107ec0 0x7f5f5c1101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c1b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1b9c50 con 0x7f5f5c119a40 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.872+0000 7f5f60dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1b9dc0 con 0x7f5f5c107af0 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.873+0000 7f5f60dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1b9f80 con 0x7f5f5c107ec0 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.873+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.873+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:49858/0 (socket says 172.21.15.6:49858) 2023-11-17T00:15:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.873+0000 7f5f5a59c700 1 -- 172.21.15.6:0/2388711989 learned_addr learned my addr 172.21.15.6:0/2388711989 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c1b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5a59c700 1 -- 172.21.15.6:0/2388711989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f5c107ec0 msgr2=0x7f5f5c1101e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f5c107ec0 0x7f5f5c1101e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5a59c700 1 -- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 msgr2=0x7f5f5c1b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c1b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.874+0000 7f5f5a59c700 1 -- 172.21.15.6:0/2388711989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44009520 con 0x7f5f5c107af0 2023-11-17T00:15:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.877+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b550 tx=0x7f5f5400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.880+0000 7f5f537fe700 1 -- 172.21.15.6:0/2388711989 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54011830 con 0x7f5f5c107af0 2023-11-17T00:15:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.880+0000 7f5f537fe700 1 -- 172.21.15.6:0/2388711989 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f54011e50 con 0x7f5f5c107af0 2023-11-17T00:15:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.880+0000 7f5f537fe700 1 -- 172.21.15.6:0/2388711989 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400f2e0 con 0x7f5f5c107af0 2023-11-17T00:15:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.880+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c11f120 con 0x7f5f5c107af0 2023-11-17T00:15:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.881+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c11ef90 con 0x7f5f5c107af0 2023-11-17T00:15:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.888+0000 7f5f537fe700 1 -- 172.21.15.6:0/2388711989 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5f5400f440 con 0x7f5f5c107af0 2023-11-17T00:15:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.891+0000 7f5f537fe700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 0x7f5f480a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.891+0000 7f5f537fe700 1 -- 172.21.15.6:0/2388711989 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f540ccf80 con 0x7f5f5c107af0 2023-11-17T00:15:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.893+0000 7f5f7477ab80 1 Processor -- start 2023-11-17T00:15:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 0x7f5f480a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 -- start start 2023-11-17T00:15:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9644fa6b0 0x55d9646a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d96469e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9645a4000 con 0x55d9646a4720 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9645f6840 con 0x55d9644fa6b0 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.894+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9645efcd0 con 0x55d9646a4b90 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51584/0 (socket says 172.21.15.6:51584) 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 -- 172.21.15.6:0/1331233808 learned_addr learned my addr 172.21.15.6:0/1331233808 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d96469e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9644fa6b0 0x55d9646a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 -- 172.21.15.6:0/1331233808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9644fa6b0 msgr2=0x55d9646a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9644fa6b0 0x55d9646a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 -- 172.21.15.6:0/1331233808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 msgr2=0x55d96469e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d96469e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.895+0000 7f5f5ad9d700 1 -- 172.21.15.6:0/1331233808 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9644715d0 con 0x55d9646a4b90 2023-11-17T00:15:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.896+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009230 tx=0x7f5f4400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf58a32ecb239dc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.897+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 0x7f5f480a3980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c1112b0 tx=0x7f5f4c0099e0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.899+0000 7f5f50ff9700 1 -- 172.21.15.6:0/1331233808 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4401c070 con 0x55d9646a4b90 2023-11-17T00:15:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.899+0000 7f5f50ff9700 1 -- 172.21.15.6:0/1331233808 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f4400f9a0 con 0x55d9646a4b90 2023-11-17T00:15:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.899+0000 7f5f50ff9700 1 -- 172.21.15.6:0/1331233808 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44005400 con 0x55d9646a4b90 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.900+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1331233808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 msgr2=0x55d96469eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.900+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009230 tx=0x7f5f4400bef0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.901+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1331233808 shutdown_connections 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.901+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9646a4b90 0x55d96469eb00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.901+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9644fa6b0 0x55d9646a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.901+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1331233808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d96469e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.901+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1331233808 >> 172.21.15.6:0/1331233808 conn(0x55d964693f50 msgr2=0x55d964699270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.902+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1331233808 shutdown_connections 2023-11-17T00:15:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.902+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1331233808 wait complete. 2023-11-17T00:15:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.902+0000 7f5f7477ab80 1 -- start start 2023-11-17T00:15:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.903+0000 7f5f7477ab80 1 Processor -- start 2023-11-17T00:15:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d9646adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d964591880 con 0x55d9646a4720 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d964591520 con 0x55d9646a4b90 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9644738c0 con 0x55d9644fa6b0 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.904+0000 7f5f7477ab80 10 client.0 _ll_register_callbacks cb 0x55d9645af720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51586/0 (socket says 172.21.15.6:51586) 2023-11-17T00:15:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1787321643 learned_addr learned my addr 172.21.15.6:0/1787321643 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d9646adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 msgr2=0x55d9646a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1787321643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 msgr2=0x55d9646adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d9646adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.905+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1787321643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c11c6e0 con 0x55d9644fa6b0 2023-11-17T00:15:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.906+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d9646adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:15:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.906+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f540967d0 tx=0x7f5f5409dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c15e679f3704f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.906+0000 7f5f41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:15:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.907+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5409a1c0 con 0x55d9644fa6b0 2023-11-17T00:15:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.907+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f54099250 con 0x55d9644fa6b0 2023-11-17T00:15:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.907+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d964691710 con 0x55d9644fa6b0 2023-11-17T00:15:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.909+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f54099410 con 0x55d9644fa6b0 2023-11-17T00:15:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.909+0000 7f5f7477ab80 10 client.26527 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.909+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d9645b6130 con 0x55d9644fa6b0 2023-11-17T00:15:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d964691710 con 0x55d9644fa6b0 2023-11-17T00:15:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f54018be0 con 0x55d9644fa6b0 2023-11-17T00:15:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 10 client.26527 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 20 client.26527 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d9646b7e80 0x55d9646ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 4 client.26527 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.910+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9645b2040 con 0x55d9646b7e80 2023-11-17T00:15:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.914+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d9646b7e80 0x55d9646ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.914+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d9646b7e80 0x55d9646ba290 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725446622c623bd8 server_cookie=9dd3d3030437f54d in_seq=0 out_seq=0 2023-11-17T00:15:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.915+0000 7f5f41ffb700 10 client.26527 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.917+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1787321643 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9645b2040 con 0x55d9646b7e80 2023-11-17T00:15:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.917+0000 7f5f41ffb700 10 client.26527 handle_command_reply: tid=0 2023-11-17T00:15:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.919+0000 7f5f7477ab80 1 client.26527 shutdown 2023-11-17T00:15:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.919+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 msgr2=0x55d9646ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.919+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f540967d0 tx=0x7f5f5409dd90 comp rx=0 tx=0).stop 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 shutdown_connections 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ada10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d9646b7e80 0x55d9646ba290 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1787321643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d9646adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 >> 172.21.15.6:0/1787321643 conn(0x55d964693f50 msgr2=0x55d9646987e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f41ffb700 0 client.26527 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.922+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 shutdown_connections 2023-11-17T00:15:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.923+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1787321643 wait complete. 2023-11-17T00:15:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.925+0000 7f5f7477ab80 20 client.26527 trim_cache size 0 max 16384 2023-11-17T00:15:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.966+0000 7f5f7477ab80 1 Processor -- start 2023-11-17T00:15:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 -- start start 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d964698f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9646958e0 con 0x55d9646a4720 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9646a41e0 con 0x55d9646a4b90 2023-11-17T00:15:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.967+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9646a6200 con 0x55d9644fa6b0 2023-11-17T00:15:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51590/0 (socket says 172.21.15.6:51590) 2023-11-17T00:15:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 -- 172.21.15.6:0/744525383 learned_addr learned my addr 172.21.15.6:0/744525383 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 -- 172.21.15.6:0/744525383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 msgr2=0x55d9646a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 -- 172.21.15.6:0/744525383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 msgr2=0x55d964698f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d964698f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.968+0000 7f5f5a59c700 1 -- 172.21.15.6:0/744525383 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9646a8900 con 0x55d9644fa6b0 2023-11-17T00:15:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.969+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400bf90 tx=0x7f5f540ada60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31222e3bb18a8565 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:30 smithi006 ceph-mon[97840]: pgmap v203: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 576 op/s 2023-11-17T00:15:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.971+0000 7f5f517fa700 1 -- 172.21.15.6:0/744525383 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400d040 con 0x55d9644fa6b0 2023-11-17T00:15:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.971+0000 7f5f517fa700 1 -- 172.21.15.6:0/744525383 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f540998a0 con 0x55d9644fa6b0 2023-11-17T00:15:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.971+0000 7f5f517fa700 1 -- 172.21.15.6:0/744525383 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f540a1d20 con 0x55d9644fa6b0 2023-11-17T00:15:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.972+0000 7f5f7477ab80 1 -- 172.21.15.6:0/744525383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 msgr2=0x55d9646a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.972+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400bf90 tx=0x7f5f540ada60 comp rx=0 tx=0).stop 2023-11-17T00:15:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.973+0000 7f5f7477ab80 1 -- 172.21.15.6:0/744525383 shutdown_connections 2023-11-17T00:15:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.973+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4b90 0x55d9646a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.973+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646a56a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.973+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/744525383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4720 0x55d964698f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.973+0000 7f5f7477ab80 1 -- 172.21.15.6:0/744525383 >> 172.21.15.6:0/744525383 conn(0x55d964693f50 msgr2=0x55d9646a4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.975+0000 7f5f7477ab80 1 -- 172.21.15.6:0/744525383 shutdown_connections 2023-11-17T00:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.975+0000 7f5f7477ab80 1 -- 172.21.15.6:0/744525383 wait complete. 2023-11-17T00:15:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.977+0000 7f5f7477ab80 1 -- start start 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.977+0000 7f5f7477ab80 1 Processor -- start 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4720 0x55d9646abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4b90 0x55d9646ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96470ad80 con 0x55d9646a4b90 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96470aef0 con 0x55d9646a4720 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96470b0b0 con 0x55d9644fa6b0 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.979+0000 7f5f7477ab80 10 client.0 _ll_register_callbacks cb 0x7f5f54010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.980+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.980+0000 7f5f59d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4720 0x55d9646abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.980+0000 7f5f5a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:51596/0 (socket says 172.21.15.6:51596) 2023-11-17T00:15:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.980+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1272248706 learned_addr learned my addr 172.21.15.6:0/1272248706 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.980+0000 7f5f5ad9d700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4b90 0x55d9646ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4720 msgr2=0x55d9646abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4720 0x55d9646abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1272248706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4b90 msgr2=0x55d9646ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4b90 0x55d9646ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 -- 172.21.15.6:0/1272248706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9645b6130 con 0x55d9644fa6b0 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f5a59c700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5f540980b0 tx=0x7f5f54099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b02d82fe374c3e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.981+0000 7f5f41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:15:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.983+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54018df0 con 0x55d9644fa6b0 2023-11-17T00:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.983+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f540a1d20 con 0x55d9644fa6b0 2023-11-17T00:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.986+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f4400f9a0 con 0x55d9644fa6b0 2023-11-17T00:15:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.986+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f540b2460 con 0x55d9644fa6b0 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.988+0000 7f5f7477ab80 10 client.26533 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.988+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d9645b5e80 con 0x55d9644fa6b0 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.988+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f4400f9a0 con 0x55d9644fa6b0 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.988+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f540c0020 con 0x55d9644fa6b0 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.989+0000 7f5f7477ab80 10 client.26533 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.989+0000 7f5f7477ab80 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.989+0000 7f5f7477ab80 20 client.26533 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.989+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d96470c2f0 0x55d96470e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.990+0000 7f5f7477ab80 4 client.26533 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:15:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.990+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9645be600 con 0x55d96470c2f0 2023-11-17T00:15:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.990+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d96470c2f0 0x55d96470e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.993+0000 7f5f59d9b700 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d96470c2f0 0x55d96470e6d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f31977d04db74d7 server_cookie=3bf2444e4cfcecb in_seq=0 out_seq=0 2023-11-17T00:15:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:30.993+0000 7f5f41ffb700 10 client.26533 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:32.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:32 smithi087 ceph-mon[131182]: pgmap v204: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 839 op/s 2023-11-17T00:15:32.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:32 smithi186 ceph-mon[131487]: pgmap v204: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 839 op/s 2023-11-17T00:15:32.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:32 smithi006 ceph-mon[97840]: pgmap v204: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 839 op/s 2023-11-17T00:15:34.867 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:34 smithi186 ceph-mon[131487]: pgmap v205: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2023-11-17T00:15:34.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:34 smithi006 ceph-mon[97840]: pgmap v205: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2023-11-17T00:15:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:34 smithi087 ceph-mon[131182]: pgmap v205: 129 pgs: 129 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 628 op/s 2023-11-17T00:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.444+0000 7f5f41ffb700 1 -- 172.21.15.6:0/1272248706 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9645be600 con 0x55d96470c2f0 2023-11-17T00:15:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.444+0000 7f5f41ffb700 10 client.26533 handle_command_reply: tid=0 2023-11-17T00:15:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.444+0000 7f5f7477ab80 1 client.26533 shutdown 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.447+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 msgr2=0x55d9646ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.447+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5f540980b0 tx=0x7f5f54099e30 comp rx=0 tx=0).stop 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 shutdown_connections 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55d96470c2f0 0x55d96470e6d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d9646a4b90 0x55d9646ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d9644fa6b0 0x55d9646ab5c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 --2- 172.21.15.6:0/1272248706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d9646a4720 0x55d9646abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 >> 172.21.15.6:0/1272248706 conn(0x55d964693f50 msgr2=0x55d9646a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.448+0000 7f5f41ffb700 0 client.26533 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.450+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 shutdown_connections 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.450+0000 7f5f7477ab80 1 -- 172.21.15.6:0/1272248706 wait complete. 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.450+0000 7f5f7477ab80 20 client.26533 trim_cache size 0 max 16384 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:15:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:15:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.451+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 msgr2=0x7f5f480a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.451+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 0x7f5f480a3980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c1112b0 tx=0x7f5f4c0099e0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.451+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 msgr2=0x7f5f5c10fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.451+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b550 tx=0x7f5f5400b910 comp rx=0 tx=0).stop 2023-11-17T00:15:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 shutdown_connections 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5f480a14d0 0x7f5f480a3980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f5c119a40 0x7f5f5c1b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f5c107af0 0x7f5f5c10fca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 --2- 172.21.15.6:0/2388711989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f5c107ec0 0x7f5f5c1101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 >> 172.21.15.6:0/2388711989 conn(0x7f5f5c06c490 msgr2=0x7f5f5c071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.453+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 shutdown_connections 2023-11-17T00:15:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:35.454+0000 7f5f60dff700 1 -- 172.21.15.6:0/2388711989 wait complete. 2023-11-17T00:15:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:36 smithi006 ceph-mon[97840]: pgmap v206: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 1.32k op/s 2023-11-17T00:15:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:36 smithi087 ceph-mon[131182]: pgmap v206: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 1.32k op/s 2023-11-17T00:15:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:36 smithi186 ceph-mon[131487]: pgmap v206: 129 pgs: 129 active+clean; 57 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 1.32k op/s 2023-11-17T00:15:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:38 smithi006 ceph-mon[97840]: pgmap v207: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:38 smithi087 ceph-mon[131182]: pgmap v207: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:39.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:38 smithi186 ceph-mon[131487]: pgmap v207: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:39.361 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 -- 172.21.15.6:0/4001664121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558074400 msgr2=0x7f55580747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 --2- 172.21.15.6:0/4001664121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558074400 0x7f55580747d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f554c009990 tx=0x7f554c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 -- 172.21.15.6:0/4001664121 shutdown_connections 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 --2- 172.21.15.6:0/4001664121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 --2- 172.21.15.6:0/4001664121 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558074400 0x7f55580747d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 --2- 172.21.15.6:0/4001664121 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558072d20 0x7f5558073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 -- 172.21.15.6:0/4001664121 >> 172.21.15.6:0/4001664121 conn(0x7f5558077500 msgr2=0x7f5558077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 -- 172.21.15.6:0/4001664121 shutdown_connections 2023-11-17T00:15:39.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.813+0000 7f555edff700 1 -- 172.21.15.6:0/4001664121 wait complete. 2023-11-17T00:15:39.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.814+0000 7f555edff700 1 Processor -- start 2023-11-17T00:15:39.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.814+0000 7f555edff700 1 -- start start 2023-11-17T00:15:39.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.814+0000 7f555edff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555edff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555edff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558074400 0x7f55581b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555edff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581b1420 con 0x7f5558074400 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555edff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581b1590 con 0x7f5558072d20 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555edff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581b1750 con 0x7f55580736d0 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52304/0 (socket says 172.21.15.6:52304) 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 -- 172.21.15.6:0/3096860702 learned_addr learned my addr 172.21.15.6:0/3096860702 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:39.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f5557fff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555d39c700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558074400 0x7f55581b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 -- 172.21.15.6:0/3096860702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 msgr2=0x7f5558105e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558105e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 -- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558074400 msgr2=0x7f55581b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558074400 0x7f55581b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f555cb9b700 1 -- 172.21.15.6:0/3096860702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544009520 con 0x7f5558072d20 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.815+0000 7f5557fff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f555cb9b700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f554c005910 tx=0x7f554c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f5555ffb700 1 -- 172.21.15.6:0/3096860702 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c01d070 con 0x7f5558072d20 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f5555ffb700 1 -- 172.21.15.6:0/3096860702 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c022470 con 0x7f5558072d20 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f5555ffb700 1 -- 172.21.15.6:0/3096860702 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c017650 con 0x7f5558072d20 2023-11-17T00:15:39.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0095f0 con 0x7f5558072d20 2023-11-17T00:15:39.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.816+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5558116990 con 0x7f5558072d20 2023-11-17T00:15:39.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.820+0000 7f5555ffb700 1 -- 172.21.15.6:0/3096860702 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f554c00fad0 con 0x7f5558072d20 2023-11-17T00:15:39.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.821+0000 7f5555ffb700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 0x7f55480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.821+0000 7f5555ffb700 1 -- 172.21.15.6:0/3096860702 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f554c0ce510 con 0x7f5558072d20 2023-11-17T00:15:39.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.821+0000 7f5572668b80 1 Processor -- start 2023-11-17T00:15:39.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.821+0000 7f5557fff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 0x7f55480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 -- start start 2023-11-17T00:15:39.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d1470 0x555a7c2d5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d58b0 0x555a7c2d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c1d6060 con 0x555a7c2d1470 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c2288a0 con 0x555a7c2d0b60 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5572668b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c221d30 con 0x555a7c2d58b0 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.822+0000 7f5557fff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 0x7f55480a37f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5544009fd0 tx=0x7f5544009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52312/0 (socket says 172.21.15.6:52312) 2023-11-17T00:15:39.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 -- 172.21.15.6:0/1715149575 learned_addr learned my addr 172.21.15.6:0/1715149575 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555d39c700 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d58b0 0x555a7c2d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f5557fff700 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d1470 0x555a7c2d5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 -- 172.21.15.6:0/1715149575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d58b0 msgr2=0x555a7c2d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d58b0 0x555a7c2d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 -- 172.21.15.6:0/1715149575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d1470 msgr2=0x555a7c2d5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d1470 0x555a7c2d5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 -- 172.21.15.6:0/1715149575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7c0a3630 con 0x555a7c2d0b60 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.823+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f554c097770 tx=0x7f554c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44e093ced4b56b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.824+0000 7f5542ffd700 1 -- 172.21.15.6:0/1715149575 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c09f2a0 con 0x555a7c2d0b60 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.824+0000 7f5542ffd700 1 -- 172.21.15.6:0/1715149575 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c0a3630 con 0x555a7c2d0b60 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.824+0000 7f5542ffd700 1 -- 172.21.15.6:0/1715149575 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0a1190 con 0x555a7c2d0b60 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.824+0000 7f5572668b80 1 -- 172.21.15.6:0/1715149575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 msgr2=0x555a7c2d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.824+0000 7f5572668b80 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f554c097770 tx=0x7f554c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 -- 172.21.15.6:0/1715149575 shutdown_connections 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d58b0 0x555a7c2d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2d0f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 --2- 172.21.15.6:0/1715149575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d1470 0x555a7c2d5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 -- 172.21.15.6:0/1715149575 >> 172.21.15.6:0/1715149575 conn(0x555a7c2c6360 msgr2=0x555a7c2cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 -- 172.21.15.6:0/1715149575 shutdown_connections 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 -- 172.21.15.6:0/1715149575 wait complete. 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.825+0000 7f5572668b80 1 -- start start 2023-11-17T00:15:39.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 Processor -- start 2023-11-17T00:15:39.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2da200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c1c38e0 con 0x555a7c2d58b0 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c1c3580 con 0x555a7c2d0b60 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.826+0000 7f5572668b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c0a5920 con 0x555a7c2d1470 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f5572668b80 10 client.0 _ll_register_callbacks cb 0x555a7c1e1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52322/0 (socket says 172.21.15.6:52322) 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 -- 172.21.15.6:0/1782191485 learned_addr learned my addr 172.21.15.6:0/1782191485 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f5557fff700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2da200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555d39c700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 -- 172.21.15.6:0/1782191485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 msgr2=0x555a7c2da200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2da200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 -- 172.21.15.6:0/1782191485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 msgr2=0x555a7c2da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2da7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 -- 172.21.15.6:0/1782191485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5558116990 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f5557fff700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2da200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f554c097720 tx=0x7f554c0cb880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fd612259f53041 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.827+0000 7f5537fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0a1070 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f554c022850 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a7c2c3b20 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f554c09a760 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 10 client.26582 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555a7c1e8190 con 0x555a7c2d0b60 2023-11-17T00:15:39.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a7c2c3b20 con 0x555a7c2d0b60 2023-11-17T00:15:39.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f554c0b3bc0 con 0x555a7c2d0b60 2023-11-17T00:15:39.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 10 client.26582 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:39.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 10 client.26582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:39.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 20 client.26582 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c2ea0b0 0x555a7c2ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 4 client.26582 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.828+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a7c1e40a0 con 0x555a7c2ea0b0 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.829+0000 7f5557fff700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c2ea0b0 0x555a7c2ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.830+0000 7f5557fff700 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c2ea0b0 0x555a7c2ec490 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5661ad6df3fccb00 server_cookie=9131bdd4ec873f0c in_seq=0 out_seq=0 2023-11-17T00:15:39.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.830+0000 7f5537fff700 10 client.26582 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5537fff700 1 -- 172.21.15.6:0/1782191485 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a7c1e40a0 con 0x555a7c2ea0b0 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5537fff700 10 client.26582 handle_command_reply: tid=0 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 client.26582 shutdown 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 msgr2=0x555a7c2dcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f554c097720 tx=0x7f554c0cb880 comp rx=0 tx=0).stop 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 shutdown_connections 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c2ea0b0 0x555a7c2ec490 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2dcf80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 --2- 172.21.15.6:0/1782191485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2da200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 >> 172.21.15.6:0/1782191485 conn(0x555a7c2c6360 msgr2=0x555a7c2d6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.835+0000 7f5537fff700 0 client.26582 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:39.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.836+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 shutdown_connections 2023-11-17T00:15:39.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.836+0000 7f5572668b80 1 -- 172.21.15.6:0/1782191485 wait complete. 2023-11-17T00:15:39.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.836+0000 7f5572668b80 20 client.26582 trim_cache size 0 max 16384 2023-11-17T00:15:39.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 Processor -- start 2023-11-17T00:15:39.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 -- start start 2023-11-17T00:15:39.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c2cd4a0 con 0x555a7c2d58b0 2023-11-17T00:15:39.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c2d19a0 con 0x555a7c2d0b60 2023-11-17T00:15:39.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.855+0000 7f5572668b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c2d1bc0 con 0x555a7c2d1470 2023-11-17T00:15:39.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f555d39c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53682/0 (socket says 172.21.15.6:53682) 2023-11-17T00:15:39.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 -- 172.21.15.6:0/2594238170 learned_addr learned my addr 172.21.15.6:0/2594238170 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:39.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f555cb9b700 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 -- 172.21.15.6:0/2594238170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 msgr2=0x555a7c2e1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2e1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 -- 172.21.15.6:0/2594238170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 msgr2=0x555a7c2d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 -- 172.21.15.6:0/2594238170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7c1e7ee0 con 0x555a7c2d1470 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.856+0000 7f5557fff700 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5558077c30 tx=0x7f5544014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d73722fd6e87c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.857+0000 7f55437fe700 1 -- 172.21.15.6:0/2594238170 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544022040 con 0x555a7c2d1470 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.857+0000 7f55437fe700 1 -- 172.21.15.6:0/2594238170 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5544028420 con 0x555a7c2d1470 2023-11-17T00:15:39.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.857+0000 7f55437fe700 1 -- 172.21.15.6:0/2594238170 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554401dc40 con 0x555a7c2d1470 2023-11-17T00:15:39.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 -- 172.21.15.6:0/2594238170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 msgr2=0x555a7c2dcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5558077c30 tx=0x7f5544014000 comp rx=0 tx=0).stop 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 -- 172.21.15.6:0/2594238170 shutdown_connections 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c2d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d0b60 0x555a7c2e1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 --2- 172.21.15.6:0/2594238170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d1470 0x555a7c2dcb20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 -- 172.21.15.6:0/2594238170 >> 172.21.15.6:0/2594238170 conn(0x555a7c2c6360 msgr2=0x555a7c2d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 -- 172.21.15.6:0/2594238170 shutdown_connections 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.858+0000 7f5572668b80 1 -- 172.21.15.6:0/2594238170 wait complete. 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.859+0000 7f5572668b80 1 -- start start 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.859+0000 7f5572668b80 1 Processor -- start 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.859+0000 7f5572668b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d0b60 0x555a7c2dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.859+0000 7f5572668b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.859+0000 7f5572668b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c33ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5572668b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c2de430 con 0x555a7c2d58b0 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5572668b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c33d400 con 0x555a7c2d1470 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5572668b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7c33d5c0 con 0x555a7c2d0b60 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5572668b80 10 client.0 _ll_register_callbacks cb 0x7f5550009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:52342/0 (socket says 172.21.15.6:52342) 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 -- 172.21.15.6:0/1631811517 learned_addr learned my addr 172.21.15.6:0/1631811517 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d0b60 0x555a7c2dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f555d39c700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c33ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 -- 172.21.15.6:0/1631811517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d0b60 msgr2=0x555a7c2dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d0b60 0x555a7c2dd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.860+0000 7f5557fff700 1 -- 172.21.15.6:0/1631811517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 msgr2=0x555a7c33ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5557fff700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c33ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5557fff700 1 -- 172.21.15.6:0/1631811517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5544017050 con 0x555a7c2d1470 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f555d39c700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c33ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5557fff700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f554401bf40 tx=0x7f554401b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cbc893ec8d1148 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5537fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544022070 con 0x555a7c2d1470 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5544005040 con 0x555a7c2d1470 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f554c0a3630 con 0x555a7c2d1470 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.861+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5544005200 con 0x555a7c2d1470 2023-11-17T00:15:39.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 10 client.26588 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555a7c038c60 con 0x555a7c2d1470 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c0a3630 con 0x555a7c2d1470 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5544037020 con 0x555a7c2d1470 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 10 client.26588 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 20 client.26588 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c33ebb0 0x555a7c340fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 4 client.26588 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:15:39.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a7c1f0660 con 0x555a7c33ebb0 2023-11-17T00:15:39.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.862+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c33ebb0 0x555a7c340fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:39.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.863+0000 7f555cb9b700 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c33ebb0 0x555a7c340fc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb284caaf921a0c server_cookie=12308a1598b13f06 in_seq=0 out_seq=0 2023-11-17T00:15:39.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.863+0000 7f5537fff700 10 client.26588 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.867+0000 7f5537fff700 1 -- 172.21.15.6:0/1631811517 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x555a7c1f0660 con 0x555a7c33ebb0 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.867+0000 7f5537fff700 10 client.26588 handle_command_reply: tid=0 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 client.26588 shutdown 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 msgr2=0x555a7c2ddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f554401bf40 tx=0x7f554401b6d0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 shutdown_connections 2023-11-17T00:15:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555a7c33ebb0 0x555a7c340fc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555a7c2d58b0 0x555a7c33ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a7c2d0b60 0x555a7c2dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5537fff700 0 client.26588 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 --2- 172.21.15.6:0/1631811517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555a7c2d1470 0x555a7c2ddc90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 >> 172.21.15.6:0/1631811517 conn(0x555a7c2c6360 msgr2=0x555a7c2cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 shutdown_connections 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 1 -- 172.21.15.6:0/1631811517 wait complete. 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.868+0000 7f5572668b80 20 client.26588 trim_cache size 0 max 16384 2023-11-17T00:15:39.871 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "scrub active (1757 inodes in the stack)", 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "9ebf54b4-47aa-46ec-acbc-7e7519866a60": { 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "9ebf54b4-47aa-46ec-acbc-7e7519866a60", 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:15:39.872 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 msgr2=0x7f55480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 0x7f55480a37f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5544009fd0 tx=0x7f5544009260 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 msgr2=0x7f5558105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f554c005910 tx=0x7f554c005740 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 shutdown_connections 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f55480a1340 0x7f55480a37f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5558074400 0x7f55581b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5558072d20 0x7f5558105930 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 --2- 172.21.15.6:0/3096860702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55580736d0 0x7f5558105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:39.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 >> 172.21.15.6:0/3096860702 conn(0x7f5558077500 msgr2=0x7f5558112730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 shutdown_connections 2023-11-17T00:15:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:15:39.869+0000 7f555edff700 1 -- 172.21.15.6:0/3096860702 wait complete. 2023-11-17T00:15:39.892 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebf54b4-47aa-46ec-acbc-7e7519866a60 - {'path': '/', 'tag': '9ebf54b4-47aa-46ec-acbc-7e7519866a60', 'options': 'recursive,force'} 2023-11-17T00:15:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:40 smithi006 ceph-mon[97840]: pgmap v208: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:41.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:40 smithi087 ceph-mon[131182]: pgmap v208: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:40 smithi186 ceph-mon[131487]: pgmap v208: 129 pgs: 129 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 1.03k op/s 2023-11-17T00:15:42.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:42 smithi006 ceph-mon[97840]: pgmap v209: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-11-17T00:15:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:42 smithi087 ceph-mon[131182]: pgmap v209: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-11-17T00:15:43.120 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:42 smithi186 ceph-mon[131487]: pgmap v209: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2023-11-17T00:15:44.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:44 smithi006 ceph-mon[97840]: pgmap v210: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.09k op/s 2023-11-17T00:15:45.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:44 smithi087 ceph-mon[131182]: pgmap v210: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.09k op/s 2023-11-17T00:15:45.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:44 smithi186 ceph-mon[131487]: pgmap v210: 129 pgs: 129 active+clean; 18 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.09k op/s 2023-11-17T00:15:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:46 smithi006 ceph-mon[97840]: pgmap v211: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.44k op/s 2023-11-17T00:15:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:46 smithi087 ceph-mon[131182]: pgmap v211: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.44k op/s 2023-11-17T00:15:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:46 smithi186 ceph-mon[131487]: pgmap v211: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 1.44k op/s 2023-11-17T00:15:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.704+0000 7f9fba7ff700 1 -- 172.21.15.6:0/349495031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101d50 msgr2=0x7f9fb410a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.704+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/349495031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101d50 0x7f9fb410a240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b0a0 tx=0x7f9fa400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 -- 172.21.15.6:0/349495031 shutdown_connections 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/349495031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/349495031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fb410a810 0x7f9fb410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/349495031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101d50 0x7f9fb410a240 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 -- 172.21.15.6:0/349495031 >> 172.21.15.6:0/349495031 conn(0x7f9fb40763a0 msgr2=0x7f9fb40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 -- 172.21.15.6:0/349495031 shutdown_connections 2023-11-17T00:15:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.705+0000 7f9fba7ff700 1 -- 172.21.15.6:0/349495031 wait complete. 2023-11-17T00:15:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.706+0000 7f9fba7ff700 1 Processor -- start 2023-11-17T00:15:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 -- start start 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fb4101d50 0x7f9fb4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb410a810 0x7f9fb419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419c600 con 0x7f9fb4101440 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419c770 con 0x7f9fb4101d50 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.708+0000 7f9fba7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419c930 con 0x7f9fb410a810 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.709+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.709+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:36798/0 (socket says 172.21.15.6:36798) 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.709+0000 7f9fb3fff700 1 -- 172.21.15.6:0/1511549802 learned_addr learned my addr 172.21.15.6:0/1511549802 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 -- 172.21.15.6:0/1511549802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb410a810 msgr2=0x7f9fb419c000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb410a810 0x7f9fb419c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 -- 172.21.15.6:0/1511549802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fb4101d50 msgr2=0x7f9fb4198c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-17T00:15:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fb4101d50 0x7f9fb4198c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 -- 172.21.15.6:0/1511549802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4009d00 con 0x7f9fb4101440 2023-11-17T00:15:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.710+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9fac014300 tx=0x7f9fac0146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.711+0000 7f9fb17fa700 1 -- 172.21.15.6:0/1511549802 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac017070 con 0x7f9fb4101440 2023-11-17T00:15:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.711+0000 7f9fb17fa700 1 -- 172.21.15.6:0/1511549802 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fac004830 con 0x7f9fb4101440 2023-11-17T00:15:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.711+0000 7f9fb17fa700 1 -- 172.21.15.6:0/1511549802 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac005260 con 0x7f9fb4101440 2023-11-17T00:15:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.711+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb41102d0 con 0x7f9fb4101440 2023-11-17T00:15:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.712+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4110140 con 0x7f9fb4101440 2023-11-17T00:15:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.723+0000 7f9fb17fa700 1 -- 172.21.15.6:0/1511549802 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9fac010a40 con 0x7f9fb4101440 2023-11-17T00:15:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.726+0000 7f9fb17fa700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 0x7f9fa00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.727+0000 7f9fb17fa700 1 -- 172.21.15.6:0/1511549802 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9fac0d4ac0 con 0x7f9fb4101440 2023-11-17T00:15:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.727+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 0x7f9fa00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.727+0000 7f9fce17bb80 1 Processor -- start 2023-11-17T00:15:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.728+0000 7f9fce17bb80 1 -- start start 2023-11-17T00:15:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.729+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 0x7f9fa00a3870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000f80 tx=0x7f9fa4009be0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.729+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.729+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.729+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f5925b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53716/0 (socket says 172.21.15.6:53716) 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fb37fe700 1 -- 172.21.15.6:0/204345792 learned_addr learned my addr 172.21.15.6:0/204345792 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f582b000 con 0x5596f592bb90 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f587d840 con 0x5596f57816b0 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5876cd0 con 0x5596f592b720 2023-11-17T00:15:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.730+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f5925b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb37fe700 1 -- 172.21.15.6:0/204345792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 msgr2=0x5596f592b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb37fe700 1 -- 172.21.15.6:0/204345792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 msgr2=0x5596f5925b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f5925b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb37fe700 1 -- 172.21.15.6:0/204345792 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f56f85d0 con 0x5596f592b720 2023-11-17T00:15:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.731+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f5925b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.734+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b0a0 tx=0x7f9fa401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb63ce9ebdac26a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.735+0000 7f9faa7fc700 1 -- 172.21.15.6:0/204345792 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400e040 con 0x5596f592b720 2023-11-17T00:15:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.735+0000 7f9faa7fc700 1 -- 172.21.15.6:0/204345792 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa4021070 con 0x5596f592b720 2023-11-17T00:15:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.735+0000 7f9faa7fc700 1 -- 172.21.15.6:0/204345792 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa401d760 con 0x5596f592b720 2023-11-17T00:15:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.736+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 msgr2=0x5596f5925560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.736+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b0a0 tx=0x7f9fa401a000 comp rx=0 tx=0).stop 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.738+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 shutdown_connections 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.738+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f5925b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.738+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.738+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/204345792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.738+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 >> 172.21.15.6:0/204345792 conn(0x5596f591af50 msgr2=0x5596f5920270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.739+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 shutdown_connections 2023-11-17T00:15:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.740+0000 7f9fce17bb80 1 -- 172.21.15.6:0/204345792 wait complete. 2023-11-17T00:15:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.741+0000 7f9fce17bb80 1 -- start start 2023-11-17T00:15:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.742+0000 7f9fce17bb80 1 Processor -- start 2023-11-17T00:15:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.743+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 0x5596f5931af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.743+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.744+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592bb90 0x5596f592f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.744+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5818880 con 0x5596f57816b0 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.744+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5818520 con 0x5596f592bb90 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.744+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f56fa8c0 con 0x5596f592b720 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.744+0000 7f9fce17bb80 10 client.0 _ll_register_callbacks cb 0x5596f5836720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 0x5596f5931af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53720/0 (socket says 172.21.15.6:53720) 2023-11-17T00:15:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 -- 172.21.15.6:0/2535083102 learned_addr learned my addr 172.21.15.6:0/2535083102 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 -- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592bb90 msgr2=0x5596f592f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592bb90 0x5596f592f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 -- 172.21.15.6:0/2535083102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 msgr2=0x5596f5931af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 0x5596f5931af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.745+0000 7f9fb37fe700 1 -- 172.21.15.6:0/2535083102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb410d8f0 con 0x5596f592b720 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.746+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 0x5596f5931af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.746+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4003c60 tx=0x7f9fa4003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bd8dd0065fcc86 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.747+0000 7f9f937fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.747+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400e040 con 0x5596f592b720 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.747+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9fa401d680 con 0x5596f592b720 2023-11-17T00:15:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.748+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596f5918710 con 0x5596f592b720 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.749+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fa401d840 con 0x5596f592b720 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.750+0000 7f9fce17bb80 10 client.26560 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.750+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5596f583d130 con 0x5596f592b720 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.752+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596f5918710 con 0x5596f592b720 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.752+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9fa403b070 con 0x5596f592b720 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.752+0000 7f9fce17bb80 10 client.26560 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.752+0000 7f9fce17bb80 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.752+0000 7f9fce17bb80 20 client.26560 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.754+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f593ec80 0x5596f5941090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.754+0000 7f9fce17bb80 4 client.26560 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:15:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.754+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f5839040 con 0x5596f593ec80 2023-11-17T00:15:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.756+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f593ec80 0x5596f5941090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.757+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f593ec80 0x5596f5941090 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e136cd59a4efe185 server_cookie=cb645625a3197875 in_seq=0 out_seq=0 2023-11-17T00:15:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.757+0000 7f9f937fe700 10 client.26560 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.760+0000 7f9f937fe700 1 -- 172.21.15.6:0/2535083102 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596f5839040 con 0x5596f593ec80 2023-11-17T00:15:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.760+0000 7f9f937fe700 10 client.26560 handle_command_reply: tid=0 2023-11-17T00:15:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.760+0000 7f9fce17bb80 1 client.26560 shutdown 2023-11-17T00:15:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.761+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 msgr2=0x5596f592ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.761+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4003c60 tx=0x7f9fa4003c90 comp rx=0 tx=0).stop 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 shutdown_connections 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f593ec80 0x5596f5941090 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f57816b0 0x5596f5931af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592bb90 0x5596f592f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/2535083102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592b720 0x5596f592ed00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9f937fe700 0 client.26560 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 >> 172.21.15.6:0/2535083102 conn(0x5596f591af50 msgr2=0x5596f591f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.762+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 shutdown_connections 2023-11-17T00:15:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.766+0000 7f9fce17bb80 1 -- 172.21.15.6:0/2535083102 wait complete. 2023-11-17T00:15:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.769+0000 7f9fce17bb80 20 client.26560 trim_cache size 0 max 16384 2023-11-17T00:15:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.806+0000 7f9fce17bb80 1 Processor -- start 2023-11-17T00:15:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.807+0000 7f9fce17bb80 1 -- start start 2023-11-17T00:15:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5918310 con 0x5596f592b720 2023-11-17T00:15:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5929a00 con 0x5596f57816b0 2023-11-17T00:15:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.808+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5930190 con 0x5596f592bb90 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.809+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.809+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.809+0000 7f9fb8d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.809+0000 7f9fb8d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53736/0 (socket says 172.21.15.6:53736) 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.809+0000 7f9fb8d9c700 1 -- 172.21.15.6:0/1072495799 learned_addr learned my addr 172.21.15.6:0/1072495799 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.810+0000 7f9fb8d9c700 1 -- 172.21.15.6:0/1072495799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 msgr2=0x5596f592a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.810+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.810+0000 7f9fb8d9c700 1 -- 172.21.15.6:0/1072495799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592b720 msgr2=0x5596f5925560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.810+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.810+0000 7f9fb8d9c700 1 -- 172.21.15.6:0/1072495799 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f591c6f0 con 0x5596f592bb90 2023-11-17T00:15:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.814+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009a90 tx=0x7f9f9c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4282fd1c4163c8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.814+0000 7f9faaffd700 1 -- 172.21.15.6:0/1072495799 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c011a50 con 0x5596f592bb90 2023-11-17T00:15:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.814+0000 7f9faaffd700 1 -- 172.21.15.6:0/1072495799 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f9c011bb0 con 0x5596f592bb90 2023-11-17T00:15:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.814+0000 7f9faaffd700 1 -- 172.21.15.6:0/1072495799 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c011a50 con 0x5596f592bb90 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.815+0000 7f9fce17bb80 1 -- 172.21.15.6:0/1072495799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 msgr2=0x5596f5925aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009a90 tx=0x7f9f9c009da0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 -- 172.21.15.6:0/1072495799 shutdown_connections 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f592bb90 0x5596f5925aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f57816b0 0x5596f592a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/1072495799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592b720 0x5596f5925560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.816+0000 7f9fce17bb80 1 -- 172.21.15.6:0/1072495799 >> 172.21.15.6:0/1072495799 conn(0x5596f591af50 msgr2=0x5596f5929de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.818+0000 7f9fce17bb80 1 -- 172.21.15.6:0/1072495799 shutdown_connections 2023-11-17T00:15:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.818+0000 7f9fce17bb80 1 -- 172.21.15.6:0/1072495799 wait complete. 2023-11-17T00:15:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.819+0000 7f9fce17bb80 1 -- start start 2023-11-17T00:15:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.819+0000 7f9fce17bb80 1 Processor -- start 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 0x5596f5932810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592b720 0x5596f5932d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f59334d0 con 0x5596f592bb90 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f598e020 con 0x5596f592b720 2023-11-17T00:15:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f598e1e0 con 0x5596f57816b0 2023-11-17T00:15:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.820+0000 7f9fce17bb80 10 client.0 _ll_register_callbacks cb 0x7f9fac0a15f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:15:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.821+0000 7f9fb8d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.821+0000 7f9fb37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592b720 0x5596f5932d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.821+0000 7f9fb8d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56878/0 (socket says 172.21.15.6:56878) 2023-11-17T00:15:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.822+0000 7f9fb8d9c700 1 -- 172.21.15.6:0/477992640 learned_addr learned my addr 172.21.15.6:0/477992640 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:15:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.822+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 0x5596f5932810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb3fff700 1 -- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592b720 msgr2=0x5596f5932d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592b720 0x5596f5932d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb3fff700 1 -- 172.21.15.6:0/477992640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 msgr2=0x5596f598dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb3fff700 1 -- 172.21.15.6:0/477992640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c009770 con 0x5596f57816b0 2023-11-17T00:15:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.823+0000 7f9fb8d9c700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:15:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.825+0000 7f9f937fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:15:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.825+0000 7f9fb3fff700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 0x5596f5932810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0d1e10 tx=0x7f9fac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433d64a5da4113e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:15:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.825+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0c12b0 con 0x5596f57816b0 2023-11-17T00:15:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.825+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9fac0d2090 con 0x5596f57816b0 2023-11-17T00:15:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.826+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fa4021070 con 0x5596f57816b0 2023-11-17T00:15:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.826+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fac0d2250 con 0x5596f57816b0 2023-11-17T00:15:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.826+0000 7f9fce17bb80 10 client.26566 fetch_fsmap learned FSMap version 22 2023-11-17T00:15:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.826+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5596f56a4d80 con 0x5596f57816b0 2023-11-17T00:15:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.827+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa4021070 con 0x5596f57816b0 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9fac0bd020 con 0x5596f57816b0 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 10 client.26566 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 20 client.26566 populate_metadata read hostname 'smithi006' 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f59a4930 0x5596f59a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 4 client.26566 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.828+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596f5845600 con 0x5596f59a4930 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.829+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f59a4930 0x5596f59a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.829+0000 7f9fb37fe700 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f59a4930 0x5596f59a6d40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d0676b7b2b39e3 server_cookie=fbda4fc483f070c6 in_seq=0 out_seq=0 2023-11-17T00:15:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:48.829+0000 7f9f937fe700 10 client.26566 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:48.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:48 smithi006 ceph-mon[97840]: pgmap v212: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 792 op/s 2023-11-17T00:15:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:48 smithi087 ceph-mon[131182]: pgmap v212: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 792 op/s 2023-11-17T00:15:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:48 smithi186 ceph-mon[131487]: pgmap v212: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 792 op/s 2023-11-17T00:15:50.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:50 smithi006 ceph-mon[97840]: pgmap v213: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 713 op/s 2023-11-17T00:15:51.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:50 smithi087 ceph-mon[131182]: pgmap v213: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 713 op/s 2023-11-17T00:15:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:50 smithi186 ceph-mon[131487]: pgmap v213: 129 pgs: 129 active+clean; 26 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 713 op/s 2023-11-17T00:15:52.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:52 smithi006 ceph-mon[97840]: pgmap v214: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 864 op/s 2023-11-17T00:15:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:52 smithi087 ceph-mon[131182]: pgmap v214: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 864 op/s 2023-11-17T00:15:53.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:52 smithi186 ceph-mon[131487]: pgmap v214: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 864 op/s 2023-11-17T00:15:54.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:54 smithi006 ceph-mon[97840]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 540 op/s 2023-11-17T00:15:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:54 smithi087 ceph-mon[131182]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 540 op/s 2023-11-17T00:15:55.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:54 smithi186 ceph-mon[131487]: pgmap v215: 129 pgs: 129 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 540 op/s 2023-11-17T00:15:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.305+0000 7f9f937fe700 1 -- 172.21.15.6:0/477992640 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596f5845600 con 0x5596f59a4930 2023-11-17T00:15:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.305+0000 7f9f937fe700 10 client.26566 handle_command_reply: tid=0 2023-11-17T00:15:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.306+0000 7f9fce17bb80 1 client.26566 shutdown 2023-11-17T00:15:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.306+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 msgr2=0x5596f5932810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.306+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 0x5596f5932810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0d1e10 tx=0x7f9fac0047c0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 shutdown_connections 2023-11-17T00:15:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5596f59a4930 0x5596f59a6d40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5596f592bb90 0x5596f598dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596f57816b0 0x5596f5932810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 --2- 172.21.15.6:0/477992640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5596f592b720 0x5596f5932d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 >> 172.21.15.6:0/477992640 conn(0x5596f591af50 msgr2=0x5596f592a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9f937fe700 0 client.26566 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 shutdown_connections 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 1 -- 172.21.15.6:0/477992640 wait complete. 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.307+0000 7f9fce17bb80 20 client.26566 trim_cache size 0 max 16384 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:15:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 msgr2=0x7f9fa00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 0x7f9fa00a3870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000f80 tx=0x7f9fa4009be0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 msgr2=0x7f9fb4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:15:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9fac014300 tx=0x7f9fac0146c0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 shutdown_connections 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9fa00a13c0 0x7f9fa00a3870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9fb4101440 0x7f9fb4198710 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb410a810 0x7f9fb419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 --2- 172.21.15.6:0/1511549802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fb4101d50 0x7f9fb4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.308+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 >> 172.21.15.6:0/1511549802 conn(0x7f9fb40763a0 msgr2=0x7f9fb410b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.309+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 shutdown_connections 2023-11-17T00:15:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:15:56.309+0000 7f9fba7ff700 1 -- 172.21.15.6:0/1511549802 wait complete. 2023-11-17T00:15:56.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:56 smithi006 ceph-mon[97840]: pgmap v216: 129 pgs: 129 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 1.17k op/s 2023-11-17T00:15:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:56 smithi087 ceph-mon[131182]: pgmap v216: 129 pgs: 129 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 1.17k op/s 2023-11-17T00:15:57.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:56 smithi186 ceph-mon[131487]: pgmap v216: 129 pgs: 129 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 1.17k op/s 2023-11-17T00:15:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:15:58 smithi006 ceph-mon[97840]: pgmap v217: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 895 op/s 2023-11-17T00:15:59.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:15:58 smithi087 ceph-mon[131182]: pgmap v217: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 895 op/s 2023-11-17T00:15:59.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:15:58 smithi186 ceph-mon[131487]: pgmap v217: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 895 op/s 2023-11-17T00:16:01.106 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:00 smithi087 ceph-mon[131182]: pgmap v218: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 855 op/s 2023-11-17T00:16:01.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:00 smithi186 ceph-mon[131487]: pgmap v218: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 855 op/s 2023-11-17T00:16:01.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:00 smithi006 ceph-mon[97840]: pgmap v218: 129 pgs: 129 active+clean; 55 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 855 op/s 2023-11-17T00:16:03.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:02 smithi087 ceph-mon[131182]: pgmap v219: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.13k op/s 2023-11-17T00:16:03.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:02 smithi186 ceph-mon[131487]: pgmap v219: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.13k op/s 2023-11-17T00:16:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:02 smithi006 ceph-mon[97840]: pgmap v219: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.13k op/s 2023-11-17T00:16:05.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:04 smithi087 ceph-mon[131182]: pgmap v220: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2023-11-17T00:16:05.251 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:04 smithi006 ceph-mon[97840]: pgmap v220: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2023-11-17T00:16:05.251 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:04 smithi186 ceph-mon[131487]: pgmap v220: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2023-11-17T00:16:07.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:06 smithi087 ceph-mon[131182]: pgmap v221: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.33k op/s 2023-11-17T00:16:07.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:06 smithi186 ceph-mon[131487]: pgmap v221: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.33k op/s 2023-11-17T00:16:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:06 smithi006 ceph-mon[97840]: pgmap v221: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.33k op/s 2023-11-17T00:16:08.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:07 smithi087 ceph-mon[131182]: osdmap e96: 12 total, 12 up, 12 in 2023-11-17T00:16:08.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:07 smithi186 ceph-mon[131487]: osdmap e96: 12 total, 12 up, 12 in 2023-11-17T00:16:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:07 smithi006 ceph-mon[97840]: osdmap e96: 12 total, 12 up, 12 in 2023-11-17T00:16:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:08 smithi087 ceph-mon[131182]: pgmap v223: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:08 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:08 smithi186 ceph-mon[131487]: pgmap v223: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:09.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:08 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:09.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:08 smithi006 ceph-mon[97840]: pgmap v223: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:09.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:08 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.573+0000 7ff50199d700 1 -- 172.21.15.6:0/2226166860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc102a40 msgr2=0x7ff4fc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.573+0000 7ff50199d700 1 --2- 172.21.15.6:0/2226166860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc102a40 0x7ff4fc102ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4f401a9a0 tx=0x7ff4f401acb0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.575+0000 7ff50199d700 1 -- 172.21.15.6:0/2226166860 shutdown_connections 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.575+0000 7ff50199d700 1 --2- 172.21.15.6:0/2226166860 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc104f40 0x7ff4fc068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.575+0000 7ff50199d700 1 --2- 172.21.15.6:0/2226166860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc102a40 0x7ff4fc102ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.575+0000 7ff50199d700 1 --2- 172.21.15.6:0/2226166860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc068640 0x7ff4fc068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.575+0000 7ff50199d700 1 -- 172.21.15.6:0/2226166860 >> 172.21.15.6:0/2226166860 conn(0x7ff4fc074d00 msgr2=0x7ff4fc075100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.577+0000 7ff50199d700 1 -- 172.21.15.6:0/2226166860 shutdown_connections 2023-11-17T00:16:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.577+0000 7ff50199d700 1 -- 172.21.15.6:0/2226166860 wait complete. 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 Processor -- start 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 -- start start 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc068640 0x7ff4fc10ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 0x7ff4fc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc1a0290 con 0x7ff4fc068640 2023-11-17T00:16:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc1a03d0 con 0x7ff4fc104f40 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.579+0000 7ff50199d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc1a0510 con 0x7ff4fc102a40 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.580+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.580+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53306/0 (socket says 172.21.15.6:53306) 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.580+0000 7ff4f3fff700 1 -- 172.21.15.6:0/2622009189 learned_addr learned my addr 172.21.15.6:0/2622009189 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.580+0000 7ff50119c700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 0x7ff4fc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50119c700 1 -- 172.21.15.6:0/2622009189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 msgr2=0x7ff4fc10f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50119c700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50119c700 1 -- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc068640 msgr2=0x7ff4fc10ed20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:16:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50099b700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc068640 0x7ff4fc10ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50119c700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc068640 0x7ff4fc10ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.581+0000 7ff50119c700 1 -- 172.21.15.6:0/2622009189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f401a600 con 0x7ff4fc104f40 2023-11-17T00:16:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.582+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.583+0000 7ff50119c700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 0x7ff4fc19fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4f401a9a0 tx=0x7ff4f4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.583+0000 7ff4fa7fc700 1 -- 172.21.15.6:0/2622009189 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f402d070 con 0x7ff4fc104f40 2023-11-17T00:16:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.583+0000 7ff4fa7fc700 1 -- 172.21.15.6:0/2622009189 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4f4003680 con 0x7ff4fc104f40 2023-11-17T00:16:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.583+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc107be0 con 0x7ff4fc104f40 2023-11-17T00:16:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.583+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4fc107a50 con 0x7ff4fc104f40 2023-11-17T00:16:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.584+0000 7ff4fa7fc700 1 -- 172.21.15.6:0/2622009189 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4031ad0 con 0x7ff4fc104f40 2023-11-17T00:16:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.590+0000 7ff4fa7fc700 1 -- 172.21.15.6:0/2622009189 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff4f4031410 con 0x7ff4fc104f40 2023-11-17T00:16:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.592+0000 7ff5164edb80 1 Processor -- start 2023-11-17T00:16:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.592+0000 7ff5164edb80 1 -- start start 2023-11-17T00:16:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a240450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a140000 con 0x555f0a23acf0 2023-11-17T00:16:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a192840 con 0x555f0a0966b0 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff5164edb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a18bcd0 con 0x555f0a240990 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff50099b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff50099b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53314/0 (socket says 172.21.15.6:53314) 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a240450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.593+0000 7ff50099b700 1 -- 172.21.15.6:0/1839300682 learned_addr learned my addr 172.21.15.6:0/1839300682 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 -- 172.21.15.6:0/1839300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 msgr2=0x555f0a240450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a240450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 -- 172.21.15.6:0/1839300682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 msgr2=0x555f0a23b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 -- 172.21.15.6:0/1839300682 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f0a00d5d0 con 0x555f0a240990 2023-11-17T00:16:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.594+0000 7ff50099b700 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009bf0 tx=0x7ff4e4009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2870455326c573 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.595+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a240450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.596+0000 7ff4f2ffd700 1 -- 172.21.15.6:0/1839300682 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400eb50 con 0x555f0a240990 2023-11-17T00:16:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.596+0000 7ff4f2ffd700 1 -- 172.21.15.6:0/1839300682 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4004cf0 con 0x555f0a240990 2023-11-17T00:16:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.596+0000 7ff4f2ffd700 1 -- 172.21.15.6:0/1839300682 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011470 con 0x555f0a240990 2023-11-17T00:16:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.597+0000 7ff4fa7fc700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 0x7ff4e80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.597+0000 7ff4fa7fc700 1 -- 172.21.15.6:0/2622009189 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff4f40dea00 con 0x7ff4fc104f40 2023-11-17T00:16:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.597+0000 7ff5164edb80 1 -- 172.21.15.6:0/1839300682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 msgr2=0x555f0a23a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.597+0000 7ff5164edb80 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009bf0 tx=0x7ff4e4009f00 comp rx=0 tx=0).stop 2023-11-17T00:16:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 -- 172.21.15.6:0/1839300682 shutdown_connections 2023-11-17T00:16:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a240450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 --2- 172.21.15.6:0/1839300682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a23a7b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 0x7ff4e80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 -- 172.21.15.6:0/1839300682 >> 172.21.15.6:0/1839300682 conn(0x555f0a2301a0 msgr2=0x555f0a2353e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 -- 172.21.15.6:0/1839300682 shutdown_connections 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.598+0000 7ff5164edb80 1 -- 172.21.15.6:0/1839300682 wait complete. 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.599+0000 7ff5164edb80 1 -- start start 2023-11-17T00:16:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.599+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 0x7ff4e80a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x555f0a2362c0 tx=0x7ff4ec0092b0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.599+0000 7ff5164edb80 1 Processor -- start 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a246c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a240990 0x555f0a2445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a12d880 con 0x555f0a240990 2023-11-17T00:16:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a12d520 con 0x555f0a0966b0 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a00f8c0 con 0x555f0a23acf0 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff5164edb80 10 client.0 _ll_register_callbacks cb 0x555f0a14b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53316/0 (socket says 172.21.15.6:53316) 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff4f3fff700 1 -- 172.21.15.6:0/3407394118 learned_addr learned my addr 172.21.15.6:0/3407394118 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff50119c700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a240990 0x555f0a2445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.600+0000 7ff50099b700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a246c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 -- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 msgr2=0x555f0a246c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a246c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 -- 172.21.15.6:0/3407394118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a240990 msgr2=0x555f0a2445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a240990 0x555f0a2445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 -- 172.21.15.6:0/3407394118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e4009820 con 0x555f0a23acf0 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec017d10 tx=0x7ff4ec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8669ebaa8e38b434 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.601+0000 7ff4dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec027070 con 0x555f0a23acf0 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff4ec018bb0 con 0x555f0a23acf0 2023-11-17T00:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f0a22d960 con 0x555f0a23acf0 2023-11-17T00:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec018d70 con 0x555f0a23acf0 2023-11-17T00:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff5164edb80 10 client.26578 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.602+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555f0a152130 con 0x555f0a23acf0 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f0a22d960 con 0x555f0a23acf0 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4ec02a020 con 0x555f0a23acf0 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 10 client.26578 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 10 client.26578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 20 client.26578 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2543f0 0x555f0a256800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 4 client.26578 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f0a14e040 con 0x555f0a2543f0 2023-11-17T00:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.605+0000 7ff50099b700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2543f0 0x555f0a256800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.606+0000 7ff50099b700 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2543f0 0x555f0a256800 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfba15ddc4fe51bd server_cookie=315c7c4818557abe in_seq=0 out_seq=0 2023-11-17T00:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.606+0000 7ff4dbfff700 10 client.26578 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.608+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3407394118 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f0a14e040 con 0x555f0a2543f0 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.608+0000 7ff4dbfff700 10 client.26578 handle_command_reply: tid=0 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.608+0000 7ff5164edb80 1 client.26578 shutdown 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.609+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 msgr2=0x555f0a243ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec017d10 tx=0x7ff4ec00b040 comp rx=0 tx=0).stop 2023-11-17T00:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 shutdown_connections 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2543f0 0x555f0a256800 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a240990 0x555f0a2445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a246c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3407394118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a23acf0 0x555f0a243ea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 >> 172.21.15.6:0/3407394118 conn(0x555f0a2301a0 msgr2=0x555f0a234a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff4dbfff700 0 client.26578 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 shutdown_connections 2023-11-17T00:16:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.610+0000 7ff5164edb80 1 -- 172.21.15.6:0/3407394118 wait complete. 2023-11-17T00:16:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.612+0000 7ff5164edb80 20 client.26578 trim_cache size 0 max 16384 2023-11-17T00:16:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.649+0000 7ff5164edb80 1 Processor -- start 2023-11-17T00:16:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.649+0000 7ff5164edb80 1 -- start start 2023-11-17T00:16:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.650+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a23a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.650+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.650+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a2504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff50099b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a23a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff5164edb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a244ca0 con 0x555f0a23acf0 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff5164edb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a24b840 con 0x555f0a0966b0 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff5164edb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a240450 con 0x555f0a240990 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff4f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39220/0 (socket says 172.21.15.6:39220) 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff4f3fff700 1 -- 172.21.15.6:0/2211535330 learned_addr learned my addr 172.21.15.6:0/2211535330 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.651+0000 7ff50119c700 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a2504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 -- 172.21.15.6:0/2211535330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 msgr2=0x555f0a2504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a2504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 -- 172.21.15.6:0/2211535330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 msgr2=0x555f0a23a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a23a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 -- 172.21.15.6:0/2211535330 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f0a151e80 con 0x555f0a23acf0 2023-11-17T00:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.652+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02b8c0 tx=0x7ff4ec02bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b846a2f5e9d1c752 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.653+0000 7ff4f37fe700 1 -- 172.21.15.6:0/2211535330 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec027070 con 0x555f0a23acf0 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.653+0000 7ff4f37fe700 1 -- 172.21.15.6:0/2211535330 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec01c070 con 0x555f0a23acf0 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.654+0000 7ff5164edb80 1 -- 172.21.15.6:0/2211535330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 msgr2=0x555f0a24b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.654+0000 7ff5164edb80 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02b8c0 tx=0x7ff4ec02bb30 comp rx=0 tx=0).stop 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 -- 172.21.15.6:0/2211535330 shutdown_connections 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a240990 0x555f0a2504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a0966b0 0x555f0a23a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 --2- 172.21.15.6:0/2211535330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a24b300 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 -- 172.21.15.6:0/2211535330 >> 172.21.15.6:0/2211535330 conn(0x555f0a2301a0 msgr2=0x555f0a23b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 -- 172.21.15.6:0/2211535330 shutdown_connections 2023-11-17T00:16:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 -- 172.21.15.6:0/2211535330 wait complete. 2023-11-17T00:16:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.655+0000 7ff5164edb80 1 -- start start 2023-11-17T00:16:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.656+0000 7ff5164edb80 1 Processor -- start 2023-11-17T00:16:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.661+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 0x555f0a23d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.661+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.661+0000 7ff5164edb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 0x555f0a23c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.661+0000 7ff5164edb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a23ca90 con 0x555f0a23acf0 2023-11-17T00:16:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff50119c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 0x555f0a23c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff50119c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 0x555f0a23c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46286/0 (socket says 172.21.15.6:46286) 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff50119c700 1 -- 172.21.15.6:0/3746862211 learned_addr learned my addr 172.21.15.6:0/3746862211 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a23cc00 con 0x555f0a240990 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0a2a82b0 con 0x555f0a0966b0 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff5164edb80 10 client.0 _ll_register_callbacks cb 0x7ff4ec0178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff50099b700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 0x555f0a23d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.662+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 -- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 msgr2=0x555f0a23c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 0x555f0a23c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 -- 172.21.15.6:0/3746862211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 msgr2=0x555f0a23be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f0a152130 con 0x555f0a0966b0 2023-11-17T00:16:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.663+0000 7ff50099b700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 0x555f0a23d640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x555f0a257110 tx=0x7ff4e40051b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4d16166a9645b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.664+0000 7ff4dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:16:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.667+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4020040 con 0x555f0a0966b0 2023-11-17T00:16:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.667+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff4e4005440 con 0x555f0a0966b0 2023-11-17T00:16:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.667+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4e4004cf0 con 0x555f0a0966b0 2023-11-17T00:16:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.670+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4e4005600 con 0x555f0a0966b0 2023-11-17T00:16:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.671+0000 7ff5164edb80 10 client.26584 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.671+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555f09fa2c00 con 0x555f0a0966b0 2023-11-17T00:16:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.671+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4004cf0 con 0x555f0a0966b0 2023-11-17T00:16:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4e400c070 con 0x555f0a0966b0 2023-11-17T00:16:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 10 client.26584 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 20 client.26584 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2a9860 0x555f0a2abc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 4 client.26584 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:16:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.672+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f0a15a600 con 0x555f0a2a9860 2023-11-17T00:16:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.673+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2a9860 0x555f0a2abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.677+0000 7ff4dbfff700 10 client.26584 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:09.677+0000 7ff4f3fff700 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2a9860 0x555f0a2abc70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702d893bba64d902 server_cookie=d86197f2fcfab3af in_seq=0 out_seq=0 2023-11-17T00:16:09.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.292+0000 7f8adb9ff700 1 -- 172.21.15.6:0/93449370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad410b680 msgr2=0x7f8ad4117ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/93449370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad410b680 0x7f8ad4117ae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b0a0 tx=0x7f8acc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 -- 172.21.15.6:0/93449370 shutdown_connections 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/93449370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad410b680 0x7f8ad4117ae0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/93449370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad4110cb0 0x7f8ad4111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/93449370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410acf0 0x7f8ad410b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.293+0000 7f8adb9ff700 1 -- 172.21.15.6:0/93449370 >> 172.21.15.6:0/93449370 conn(0x7f8ad406b300 msgr2=0x7f8ad406b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.294+0000 7f8adb9ff700 1 -- 172.21.15.6:0/93449370 shutdown_connections 2023-11-17T00:16:10.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.294+0000 7f8adb9ff700 1 -- 172.21.15.6:0/93449370 wait complete. 2023-11-17T00:16:10.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.296+0000 7f8adb9ff700 1 Processor -- start 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.296+0000 7f8adb9ff700 1 -- start start 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.296+0000 7f8adb9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.296+0000 7f8adb9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 0x7f8ad4114660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8adb9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad4110cb0 0x7f8ad4114ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8adb9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad41150e0 con 0x7f8ad410b680 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8adb9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4115250 con 0x7f8ad4110cb0 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8adb9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4115410 con 0x7f8ad410acf0 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53374/0 (socket says 172.21.15.6:53374) 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 -- 172.21.15.6:0/1180571112 learned_addr learned my addr 172.21.15.6:0/1180571112 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad4110cb0 0x7f8ad4114ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 0x7f8ad4114660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 -- 172.21.15.6:0/1180571112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad4110cb0 msgr2=0x7f8ad4114ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad4110cb0 0x7f8ad4114ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 -- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 msgr2=0x7f8ad4114660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 0x7f8ad4114660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad979b700 1 -- 172.21.15.6:0/1180571112 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009d00 con 0x7f8ad410acf0 2023-11-17T00:16:10.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.297+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 0x7f8ad4114660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.298+0000 7f8ad979b700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c9b0 tx=0x7f8ad000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.299+0000 7f8aca7fc700 1 -- 172.21.15.6:0/1180571112 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0007b00 con 0x7f8ad410acf0 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.299+0000 7f8aca7fc700 1 -- 172.21.15.6:0/1180571112 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad0007c60 con 0x7f8ad410acf0 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.299+0000 7f8aca7fc700 1 -- 172.21.15.6:0/1180571112 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0018660 con 0x7f8ad410acf0 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.299+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad411ad10 con 0x7f8ad410acf0 2023-11-17T00:16:10.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.300+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad411ab80 con 0x7f8ad410acf0 2023-11-17T00:16:10.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.310+0000 7f8aef3cdb80 1 Processor -- start 2023-11-17T00:16:10.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.311+0000 7f8aef3cdb80 1 -- start start 2023-11-17T00:16:10.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.311+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x55725073ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aca7fc700 1 -- 172.21.15.6:0/1180571112 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8ad000f450 con 0x7f8ad410acf0 2023-11-17T00:16:10.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 0x557250743370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250743d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x55725073ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 0x557250743370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250644060 con 0x55725073eb60 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572506968a0 con 0x55725073f470 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725068fd30 con 0x5572507438b0 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 0x557250743370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46306/0 (socket says 172.21.15.6:46306) 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8ad979b700 1 -- 172.21.15.6:0/1895293527 learned_addr learned my addr 172.21.15.6:0/1895293527 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.312+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x55725073ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39236/0 (socket says 172.21.15.6:39236) 2023-11-17T00:16:10.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.313+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250743d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.316+0000 7f8ad8f9a700 1 -- 172.21.15.6:0/1895293527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 msgr2=0x557250743370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.316+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 0x557250743370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.316+0000 7f8ad8f9a700 1 -- 172.21.15.6:0/1895293527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 msgr2=0x55725073ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.316+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x55725073ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.316+0000 7f8ad8f9a700 1 -- 172.21.15.6:0/1895293527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557250511630 con 0x5572507438b0 2023-11-17T00:16:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.317+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250743d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ad41091d0 tx=0x7f8ac400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c35791e8496f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.319+0000 7f8abf7fe700 1 -- 172.21.15.6:0/1895293527 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400faf0 con 0x5572507438b0 2023-11-17T00:16:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.319+0000 7f8abf7fe700 1 -- 172.21.15.6:0/1895293527 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ac4004cf0 con 0x5572507438b0 2023-11-17T00:16:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.319+0000 7f8abf7fe700 1 -- 172.21.15.6:0/1895293527 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4005700 con 0x5572507438b0 2023-11-17T00:16:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.321+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/1895293527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 msgr2=0x557250743d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:10.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.321+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250743d00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8ad41091d0 tx=0x7f8ac400bf10 comp rx=0 tx=0).stop 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aca7fc700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 0x7f8ac00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aca7fc700 1 -- 172.21.15.6:0/1180571112 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8ad00cd030 con 0x7f8ad410acf0 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/1895293527 shutdown_connections 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x55725073ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250743d00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/1895293527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073f470 0x557250743370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/1895293527 >> 172.21.15.6:0/1895293527 conn(0x557250734360 msgr2=0x557250739b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.322+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 0x7f8ac00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.323+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/1895293527 shutdown_connections 2023-11-17T00:16:10.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.323+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/1895293527 wait complete. 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.323+0000 7f8aef3cdb80 1 -- start start 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.324+0000 7f8aef3cdb80 1 Processor -- start 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.327+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 0x7f8ac00a3940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b830 tx=0x7f8acc006210 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073eb60 0x557250748380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x5572507488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572506318e0 con 0x55725073f470 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250631580 con 0x55725073eb60 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250513920 con 0x5572507438b0 2023-11-17T00:16:10.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8aef3cdb80 10 client.0 _ll_register_callbacks cb 0x55725064f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:10.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53384/0 (socket says 172.21.15.6:53384) 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.328+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/740815925 learned_addr learned my addr 172.21.15.6:0/740815925 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073eb60 msgr2=0x557250748380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073eb60 0x557250748380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/740815925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 msgr2=0x5572507488c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x5572507488c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:10.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/740815925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac400b890 con 0x5572507438b0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.329+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007760 tx=0x7f8acc00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=776fb982b23b6ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.330+0000 7f8abcff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.330+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00e040 con 0x5572507438b0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.330+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8acc0054d0 con 0x5572507438b0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.330+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557250731b20 con 0x5572507438b0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.331+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8acc005690 con 0x5572507438b0 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.331+0000 7f8aef3cdb80 10 client.26593 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:10.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.331+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557250656190 con 0x5572507438b0 2023-11-17T00:16:10.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.334+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557250731b20 con 0x5572507438b0 2023-11-17T00:16:10.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.335+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8acc029ad0 con 0x5572507438b0 2023-11-17T00:16:10.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.335+0000 7f8aef3cdb80 10 client.26593 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:10.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.335+0000 7f8aef3cdb80 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:10.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.335+0000 7f8aef3cdb80 20 client.26593 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:10.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.335+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557250758960 0x55725075ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:10.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.336+0000 7f8aef3cdb80 4 client.26593 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:10.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.336+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572506520a0 con 0x557250758960 2023-11-17T00:16:10.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.337+0000 7f8ad979b700 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557250758960 0x55725075ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:10.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.339+0000 7f8ad979b700 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557250758960 0x55725075ad90 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de805ed781a72583 server_cookie=4ef1583d51ec51da in_seq=0 out_seq=0 2023-11-17T00:16:10.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:10.339+0000 7f8abcff9700 10 client.26593 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:11.107 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:10 smithi087 ceph-mon[131182]: pgmap v224: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:10 smithi186 ceph-mon[131487]: pgmap v224: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:10 smithi006 ceph-mon[97840]: pgmap v224: 129 pgs: 129 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 876 op/s 2023-11-17T00:16:12.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:11 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:16:12.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:11 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:16:12.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:11 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.518+0000 7ff4dbfff700 1 -- 172.21.15.6:0/3746862211 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f0a15a600 con 0x555f0a2a9860 2023-11-17T00:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.518+0000 7ff4dbfff700 10 client.26584 handle_command_reply: tid=0 2023-11-17T00:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.518+0000 7ff5164edb80 1 client.26584 shutdown 2023-11-17T00:16:12.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.518+0000 7f8abcff9700 1 -- 172.21.15.6:0/740815925 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572506520a0 con 0x557250758960 2023-11-17T00:16:12.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.518+0000 7f8abcff9700 10 client.26593 handle_command_reply: tid=0 2023-11-17T00:16:12.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 client.26593 shutdown 2023-11-17T00:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 msgr2=0x555f0a23d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 0x555f0a23d640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x555f0a257110 tx=0x7ff4e40051b0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 msgr2=0x557250748e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007760 tx=0x7f8acc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 shutdown_connections 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555f0a240990 0x555f0a23c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f0a0966b0 0x555f0a23d640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555f0a2a9860 0x555f0a2abc70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 --2- 172.21.15.6:0/3746862211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555f0a23acf0 0x555f0a23be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 >> 172.21.15.6:0/3746862211 conn(0x555f0a2301a0 msgr2=0x555f0a2504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff4dbfff700 0 client.26584 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 shutdown_connections 2023-11-17T00:16:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 1 -- 172.21.15.6:0/3746862211 wait complete. 2023-11-17T00:16:12.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 shutdown_connections 2023-11-17T00:16:12.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5572507438b0 0x557250748e00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55725073eb60 0x557250748380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557250758960 0x55725075ad90 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/740815925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x5572507488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8abcff9700 0 client.26593 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:12.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.519+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 >> 172.21.15.6:0/740815925 conn(0x557250734360 msgr2=0x557250744d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.519+0000 7ff5164edb80 20 client.26584 trim_cache size 0 max 16384 2023-11-17T00:16:12.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.520+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 shutdown_connections 2023-11-17T00:16:12.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.520+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/740815925 wait complete. 2023-11-17T00:16:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:16:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:16:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:16:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:16:12.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.520+0000 7f8aef3cdb80 20 client.26593 trim_cache size 0 max 16384 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 msgr2=0x7ff4e80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 0x7ff4e80a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x555f0a2362c0 tx=0x7ff4ec0092b0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 msgr2=0x7ff4fc19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 0x7ff4fc19fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4f401a9a0 tx=0x7ff4f4003b90 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 shutdown_connections 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff4e80a1340 0x7ff4e80a37f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4fc068640 0x7ff4fc10ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4fc104f40 0x7ff4fc19fd50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.520+0000 7ff50199d700 1 --2- 172.21.15.6:0/2622009189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4fc102a40 0x7ff4fc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.521+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 >> 172.21.15.6:0/2622009189 conn(0x7ff4fc074d00 msgr2=0x7ff4fc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.521+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 shutdown_connections 2023-11-17T00:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:12.521+0000 7ff50199d700 1 -- 172.21.15.6:0/2622009189 wait complete. 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 Processor -- start 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 -- start start 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 0x55725074f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x557250741970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725073fc00 con 0x55725073f470 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725074b430 con 0x5572507438b0 2023-11-17T00:16:12.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572507468e0 con 0x55725073eb60 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 0x55725074f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad8f9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x557250741970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad979b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 0x55725074f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53388/0 (socket says 172.21.15.6:53388) 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46330/0 (socket says 172.21.15.6:46330) 2023-11-17T00:16:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.539+0000 7f8ad979b700 1 -- 172.21.15.6:0/3070945874 learned_addr learned my addr 172.21.15.6:0/3070945874 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:12.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3070945874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 msgr2=0x55725074f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:12.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 0x55725074f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3070945874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 msgr2=0x557250741970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:12.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x557250741970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3070945874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557250655ee0 con 0x5572507438b0 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00a010 tx=0x7f8acc009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329cccb4de15893 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8abffff700 1 -- 172.21.15.6:0/3070945874 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00b860 con 0x5572507438b0 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8abffff700 1 -- 172.21.15.6:0/3070945874 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8acc01d420 con 0x5572507438b0 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.540+0000 7f8abffff700 1 -- 172.21.15.6:0/3070945874 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc029a40 con 0x5572507438b0 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3070945874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 msgr2=0x557250741eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00a010 tx=0x7f8acc009bc0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3070945874 shutdown_connections 2023-11-17T00:16:12.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250741eb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073eb60 0x55725074f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3070945874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073f470 0x557250741970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3070945874 >> 172.21.15.6:0/3070945874 conn(0x557250734360 msgr2=0x55725075a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3070945874 shutdown_connections 2023-11-17T00:16:12.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.541+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3070945874 wait complete. 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.542+0000 7f8aef3cdb80 1 -- start start 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.542+0000 7f8aef3cdb80 1 Processor -- start 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x5572507449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 0x557250747c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250744f00 con 0x55725073eb60 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250745070 con 0x5572507438b0 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557250745230 con 0x55725073f470 2023-11-17T00:16:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8aef3cdb80 10 client.0 _ll_register_callbacks cb 0x7f8acc003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad8f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 0x557250747c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46338/0 (socket says 172.21.15.6:46338) 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3221255649 learned_addr learned my addr 172.21.15.6:0/3221255649 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad979b700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x5572507449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3221255649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 msgr2=0x557250747c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 0x557250747c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3221255649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 msgr2=0x5572507449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x5572507449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad9f9c700 1 -- 172.21.15.6:0/3221255649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557250656190 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.543+0000 7f8ad8f9a700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 0x557250747c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8ad9f9c700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8acc01d5b0 tx=0x7f8acc009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdb9803d646bba8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8abcff9700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc029450 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8acc029c20 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ac4004cf0 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8acc050760 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8aef3cdb80 10 client.26630 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5572504a6c60 con 0x5572507438b0 2023-11-17T00:16:12.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.544+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ac4004cf0 con 0x5572507438b0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8acc005320 con 0x5572507438b0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 10 client.26630 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 20 client.26630 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5572507ad3c0 0x5572507af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 4 client.26630 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55725065e660 con 0x5572507ad3c0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.545+0000 7f8ad979b700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5572507ad3c0 0x5572507af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8abcff9700 10 client.26630 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8ad979b700 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5572507ad3c0 0x5572507af7a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552aa8dd72bfc83 server_cookie=32e941a89406ebba in_seq=0 out_seq=0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8abcff9700 1 -- 172.21.15.6:0/3221255649 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55725065e660 con 0x5572507ad3c0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8abcff9700 10 client.26630 handle_command_reply: tid=0 2023-11-17T00:16:12.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 client.26630 shutdown 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 msgr2=0x557250748160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8acc01d5b0 tx=0x7f8acc009480 comp rx=0 tx=0).stop 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 shutdown_connections 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5572507ad3c0 0x5572507af7a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55725073eb60 0x5572507449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5572507438b0 0x557250748160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8abcff9700 0 client.26630 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 --2- 172.21.15.6:0/3221255649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55725073f470 0x557250747c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 >> 172.21.15.6:0/3221255649 conn(0x557250734360 msgr2=0x557250741970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 shutdown_connections 2023-11-17T00:16:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 1 -- 172.21.15.6:0/3221255649 wait complete. 2023-11-17T00:16:12.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.546+0000 7f8aef3cdb80 20 client.26630 trim_cache size 0 max 16384 2023-11-17T00:16:12.555 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:12.555 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:16:12.557 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:16:12.557 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:12.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 msgr2=0x7f8ac00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 0x7f8ac00a3940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b830 tx=0x7f8acc006210 comp rx=0 tx=0).stop 2023-11-17T00:16:12.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 msgr2=0x7f8ad4117650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c9b0 tx=0x7f8ad000ccc0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 shutdown_connections 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f8ac00a1490 0x7f8ac00a3940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ad4110cb0 0x7f8ad4114ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ad410acf0 0x7f8ad4117650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 --2- 172.21.15.6:0/1180571112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ad410b680 0x7f8ad4114660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 >> 172.21.15.6:0/1180571112 conn(0x7f8ad406b300 msgr2=0x7f8ad410e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 shutdown_connections 2023-11-17T00:16:12.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.547+0000 7f8adb9ff700 1 -- 172.21.15.6:0/1180571112 wait complete. 2023-11-17T00:16:12.567 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:16:12.568 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:16:12.810 INFO:tasks.workunit.client.0.smithi006.stdout:cleaned up hierarchy 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.830+0000 7fede5fff700 1 -- 172.21.15.6:0/4025548285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0105620 msgr2=0x7fede010d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.830+0000 7fede5fff700 1 --2- 172.21.15.6:0/4025548285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0105620 0x7fede010d1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009fc0 tx=0x7fedd800b330 comp rx=0 tx=0).stop 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 -- 172.21.15.6:0/4025548285 shutdown_connections 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 --2- 172.21.15.6:0/4025548285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0100cc0 0x7fede0101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 --2- 172.21.15.6:0/4025548285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0105620 0x7fede010d1a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 --2- 172.21.15.6:0/4025548285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0101660 0x7fede0104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 -- 172.21.15.6:0/4025548285 >> 172.21.15.6:0/4025548285 conn(0x7fede00774d0 msgr2=0x7fede00778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 -- 172.21.15.6:0/4025548285 shutdown_connections 2023-11-17T00:16:12.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.831+0000 7fede5fff700 1 -- 172.21.15.6:0/4025548285 wait complete. 2023-11-17T00:16:12.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 Processor -- start 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 -- start start 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0101660 0x7fede0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0105620 0x7fede01a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0073b60 con 0x7fede0101660 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01a4ca0 con 0x7fede0100cc0 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7fede5fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01a4de0 con 0x7fede0105620 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7feddf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7feddf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46366/0 (socket says 172.21.15.6:46366) 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.832+0000 7feddf7fe700 1 -- 172.21.15.6:0/299305500 learned_addr learned my addr 172.21.15.6:0/299305500 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddeffd700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0101660 0x7fede0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddffff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0105620 0x7fede01a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 -- 172.21.15.6:0/299305500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0105620 msgr2=0x7fede01a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0105620 0x7fede01a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 -- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0101660 msgr2=0x7fede0073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0101660 0x7fede0073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 -- 172.21.15.6:0/299305500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8009c40 con 0x7fede0100cc0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddf7fe700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9c0 tx=0x7fedd000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0004a30 con 0x7fede0100cc0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.833+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd000cc80 con 0x7fede0100cc0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.834+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01103e0 con 0x7fede0100cc0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.834+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00054e0 con 0x7fede0100cc0 2023-11-17T00:16:12.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.834+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0110250 con 0x7fede0100cc0 2023-11-17T00:16:12.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.835+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede0110570 con 0x7fede0100cc0 2023-11-17T00:16:12.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.837+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fedd0004b90 con 0x7fede0100cc0 2023-11-17T00:16:12.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.838+0000 7feddcff9700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 0x7fedc80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:12.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.838+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fedd0014070 con 0x7fede0100cc0 2023-11-17T00:16:12.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.838+0000 7feddeffd700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 0x7fedc80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:12.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.839+0000 7feddeffd700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 0x7fedc80a3830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fede00745e0 tx=0x7fedd4009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:12.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:12.840+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fedd0096900 con 0x7fede0100cc0 2023-11-17T00:16:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:12 smithi006 ceph-mon[97840]: pgmap v225: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:12 smithi087 ceph-mon[131182]: pgmap v225: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:13.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.113+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fede0074320 con 0x7fede0100cc0 2023-11-17T00:16:13.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.114+0000 7feddcff9700 1 -- 172.21.15.6:0/299305500 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fede0074320 con 0x7fede0100cc0 2023-11-17T00:16:13.115 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:16:13.115 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:16:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:12 smithi186 ceph-mon[131487]: pgmap v225: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:13.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 msgr2=0x7fedc80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 0x7fedc80a3830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fede00745e0 tx=0x7fedd4009260 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 msgr2=0x7fede0072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9c0 tx=0x7fedd000ecd0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 shutdown_connections 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fedc80a1380 0x7fedc80a3830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fede0105620 0x7fede01a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede0100cc0 0x7fede0072df0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 --2- 172.21.15.6:0/299305500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fede0101660 0x7fede0073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 >> 172.21.15.6:0/299305500 conn(0x7fede00774d0 msgr2=0x7fede00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 shutdown_connections 2023-11-17T00:16:13.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.116+0000 7fede5fff700 1 -- 172.21.15.6:0/299305500 wait complete. 2023-11-17T00:16:13.119 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:16:13.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/1766800129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 msgr2=0x7fd2f810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/1766800129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f810d080 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000b0d0 tx=0x7fd2f000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/1766800129 shutdown_connections 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/1766800129 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f810d080 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/1766800129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f81052a0 0x7fd2f8105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/1766800129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f8067dc0 0x7fd2f8068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/1766800129 >> 172.21.15.6:0/1766800129 conn(0x7fd2f80763a0 msgr2=0x7fd2f80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/1766800129 shutdown_connections 2023-11-17T00:16:13.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.360+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/1766800129 wait complete. 2023-11-17T00:16:13.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.361+0000 7fd2fd1ff700 1 Processor -- start 2023-11-17T00:16:13.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 -- start start 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f8100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f81052a0 0x7fd2f81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81008b0 con 0x7fd2f8068770 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81a4b60 con 0x7fd2f8067dc0 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2fd1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81a4ca0 con 0x7fd2f81052a0 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.362+0000 7fd2f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46386/0 (socket says 172.21.15.6:46386) 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/463986671 learned_addr learned my addr 172.21.15.6:0/463986671 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:13.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f659c700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f8100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f759e700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f81052a0 0x7fd2f81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/463986671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f81052a0 msgr2=0x7fd2f81a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f81052a0 0x7fd2f81a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 msgr2=0x7fd2f8100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f8100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/463986671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8009520 con 0x7fd2f8067dc0 2023-11-17T00:16:13.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800eb30 tx=0x7fd2e800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2dffff700 1 -- 172.21.15.6:0/463986671 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8009880 con 0x7fd2f8067dc0 2023-11-17T00:16:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2dffff700 1 -- 172.21.15.6:0/463986671 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e80044e0 con 0x7fd2f8067dc0 2023-11-17T00:16:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f0009d00 con 0x7fd2f8067dc0 2023-11-17T00:16:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.363+0000 7fd2dffff700 1 -- 172.21.15.6:0/463986671 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003d80 con 0x7fd2f8067dc0 2023-11-17T00:16:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.364+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f8110390 con 0x7fd2f8067dc0 2023-11-17T00:16:13.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.367+0000 7fd2dffff700 1 -- 172.21.15.6:0/463986671 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd2e8003f20 con 0x7fd2f8067dc0 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.368+0000 7fd310b57b80 1 Processor -- start 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.368+0000 7fd310b57b80 1 -- start start 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.368+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55badddd4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.368+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddd9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.368+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd310b57b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddcda060 con 0x55badddd4b60 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd310b57b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddd2c8a0 con 0x55badddd5470 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd310b57b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddd25d30 con 0x55badddd98b0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2dffff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 0x7fd2e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2dffff700 1 -- 172.21.15.6:0/463986671 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd2e8014070 con 0x7fd2f8067dc0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:53438/0 (socket says 172.21.15.6:53438) 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddd9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 -- 172.21.15.6:0/3219636046 learned_addr learned my addr 172.21.15.6:0/3219636046 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f759e700 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55badddd4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f759e700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 0x7fd2e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 -- 172.21.15.6:0/3219636046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 msgr2=0x55badddd9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddd9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 -- 172.21.15.6:0/3219636046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 msgr2=0x55badddd4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55badddd4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.369+0000 7fd2f659c700 1 -- 172.21.15.6:0/3219636046 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baddba7630 con 0x55badddd98b0 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.370+0000 7fd2f659c700 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009d20 tx=0x7fd2ec011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51afeff124a5090d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.370+0000 7fd2f759e700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 0x7fd2e00a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00060b0 tx=0x7fd2f000b040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.370+0000 7fd2dd7fa700 1 -- 172.21.15.6:0/3219636046 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec019630 con 0x55badddd98b0 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.370+0000 7fd2dd7fa700 1 -- 172.21.15.6:0/3219636046 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec019c50 con 0x55badddd98b0 2023-11-17T00:16:13.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.370+0000 7fd2dd7fa700 1 -- 172.21.15.6:0/3219636046 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00ec00 con 0x55badddd98b0 2023-11-17T00:16:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- 172.21.15.6:0/3219636046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 msgr2=0x55badddd9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009d20 tx=0x7fd2ec011720 comp rx=0 tx=0).stop 2023-11-17T00:16:13.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- 172.21.15.6:0/3219636046 shutdown_connections 2023-11-17T00:16:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55badddd4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55badddd9d00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 --2- 172.21.15.6:0/3219636046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddd9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- 172.21.15.6:0/3219636046 >> 172.21.15.6:0/3219636046 conn(0x55badddca360 msgr2=0x55badddcfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- 172.21.15.6:0/3219636046 shutdown_connections 2023-11-17T00:16:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- 172.21.15.6:0/3219636046 wait complete. 2023-11-17T00:16:13.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.371+0000 7fd310b57b80 1 -- start start 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.372+0000 7fd310b57b80 1 Processor -- start 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd5470 0x55badddde240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55badddde870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddcc78e0 con 0x55badddd98b0 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddcc7580 con 0x55badddd4b60 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddba9920 con 0x55badddd5470 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd310b57b80 10 client.0 _ll_register_callbacks cb 0x55baddce5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46408/0 (socket says 172.21.15.6:46408) 2023-11-17T00:16:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/1335212691 learned_addr learned my addr 172.21.15.6:0/1335212691 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f659c700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd5470 0x55badddde240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f759e700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55badddde870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/1335212691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd5470 msgr2=0x55badddde240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd5470 0x55badddde240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/1335212691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 msgr2=0x55badddde870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55badddde870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.373+0000 7fd2f6d9d700 1 -- 172.21.15.6:0/1335212691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec009980 con 0x55badddd4b60 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c5b0 tx=0x7fd2e8099d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca52abd0c9ed9bb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80b6930 con 0x55badddd4b60 2023-11-17T00:16:13.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd2e800cbd0 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55badddc7b20 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e809e170 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 10 client.26648 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55baddcec190 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55badddc7b20 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2e80cd360 con 0x55badddd4b60 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 10 client.26648 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 20 client.26648 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badddee7e0 0x55badddf0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 4 client.26648 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:13.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.374+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baddce80a0 con 0x55badddee7e0 2023-11-17T00:16:13.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.375+0000 7fd2f659c700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badddee7e0 0x55badddf0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.375+0000 7fd2f659c700 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badddee7e0 0x55badddf0bf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95d402eaccf13db server_cookie=6790a284f2fafda in_seq=0 out_seq=0 2023-11-17T00:16:13.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.375+0000 7fd2daff5700 10 client.26648 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.376+0000 7fd2daff5700 1 -- 172.21.15.6:0/1335212691 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55baddce80a0 con 0x55badddee7e0 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.376+0000 7fd2daff5700 10 client.26648 handle_command_reply: tid=0 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 client.26648 shutdown 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 msgr2=0x55baddde0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c5b0 tx=0x7fd2e8099d80 comp rx=0 tx=0).stop 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 shutdown_connections 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badddee7e0 0x55badddf0bf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55badddde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd4b60 0x55baddde0fc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd2daff5700 0 client.26648 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:13.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 --2- 172.21.15.6:0/1335212691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd5470 0x55badddde240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 >> 172.21.15.6:0/1335212691 conn(0x55badddca360 msgr2=0x55baddddad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 shutdown_connections 2023-11-17T00:16:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 1 -- 172.21.15.6:0/1335212691 wait complete. 2023-11-17T00:16:13.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.377+0000 7fd310b57b80 20 client.26648 trim_cache size 0 max 16384 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.392+0000 7fd310b57b80 1 Processor -- start 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.392+0000 7fd310b57b80 1 -- start start 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 0x55baddde59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55baddde0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddddf820 con 0x55badddd98b0 2023-11-17T00:16:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badddd8180 con 0x55badddd5470 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd310b57b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badddd8370 con 0x55badddd4b60 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55baddde0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46422/0 (socket says 172.21.15.6:46422) 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 -- 172.21.15.6:0/2679785809 learned_addr learned my addr 172.21.15.6:0/2679785809 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 0x55baddde59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 -- 172.21.15.6:0/2679785809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 msgr2=0x55baddde59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 0x55baddde59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 -- 172.21.15.6:0/2679785809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 msgr2=0x55baddde0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55baddde0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f659c700 1 -- 172.21.15.6:0/2679785809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55badddda1a0 con 0x55badddd5470 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.393+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 0x55baddde59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009c70 tx=0x7fd2ec0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68edadc7f4a347b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd2ddffb700 1 -- 172.21.15.6:0/2679785809 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec029070 con 0x55badddd5470 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd2ddffb700 1 -- 172.21.15.6:0/2679785809 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec019670 con 0x55badddd5470 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd2ddffb700 1 -- 172.21.15.6:0/2679785809 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec02aa50 con 0x55badddd5470 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd310b57b80 1 -- 172.21.15.6:0/2679785809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 msgr2=0x55badddeab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.394+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009c70 tx=0x7fd2ec0058e0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 -- 172.21.15.6:0/2679785809 shutdown_connections 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd98b0 0x55baddde0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd4b60 0x55baddde59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2679785809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55badddeab80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 -- 172.21.15.6:0/2679785809 >> 172.21.15.6:0/2679785809 conn(0x55badddca360 msgr2=0x55badddce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 -- 172.21.15.6:0/2679785809 shutdown_connections 2023-11-17T00:16:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 -- 172.21.15.6:0/2679785809 wait complete. 2023-11-17T00:16:13.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 -- start start 2023-11-17T00:16:13.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.395+0000 7fd310b57b80 1 Processor -- start 2023-11-17T00:16:13.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55baddde5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55baddde3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddde5d50 con 0x55badddd4b60 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddde4410 con 0x55badddd5470 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baddde45d0 con 0x55badddd98b0 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.396+0000 7fd310b57b80 10 client.0 _ll_register_callbacks cb 0x7fd2e809a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46428/0 (socket says 172.21.15.6:46428) 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 -- 172.21.15.6:0/2387813123 learned_addr learned my addr 172.21.15.6:0/2387813123 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f759e700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55baddde3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55baddde5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 -- 172.21.15.6:0/2387813123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 msgr2=0x55baddde3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55baddde3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 -- 172.21.15.6:0/2387813123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 msgr2=0x55baddde5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55baddde5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 -- 172.21.15.6:0/2387813123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55baddba7630 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55baddde5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2f659c700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009d20 tx=0x7fd2ec019a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4e7f97cffc6a12 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2daff5700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec029040 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.397+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2ec019c50 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd2ec0054c0 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ec005680 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 10 client.26654 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55baddb53de0 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec019c50 con 0x55badddd5470 2023-11-17T00:16:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2ec00e800 con 0x55badddd5470 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 10 client.26654 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 20 client.26654 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badde43500 0x55badde458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 4 client.26654 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.398+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55baddcf4660 con 0x55badde43500 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.399+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badde43500 0x55badde458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.399+0000 7fd2f6d9d700 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badde43500 0x55badde458e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcd22d71620b904 server_cookie=74463bc98ce9318 in_seq=0 out_seq=0 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.399+0000 7fd2daff5700 10 client.26654 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:13.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd2daff5700 1 -- 172.21.15.6:0/2387813123 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55baddcf4660 con 0x55badde43500 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd2daff5700 10 client.26654 handle_command_reply: tid=0 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 client.26654 shutdown 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 msgr2=0x55baddde3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009d20 tx=0x7fd2ec019a90 comp rx=0 tx=0).stop 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 shutdown_connections 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55badde43500 0x55badde458e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55badddd4b60 0x55baddde5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55badddd98b0 0x55baddde3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd2daff5700 0 client.26654 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 --2- 172.21.15.6:0/2387813123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55badddd5470 0x55baddde3990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 >> 172.21.15.6:0/2387813123 conn(0x55badddca360 msgr2=0x55badddeab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 shutdown_connections 2023-11-17T00:16:13.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.400+0000 7fd310b57b80 1 -- 172.21.15.6:0/2387813123 wait complete. 2023-11-17T00:16:13.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd310b57b80 20 client.26654 trim_cache size 0 max 16384 2023-11-17T00:16:13.414 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:16:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 msgr2=0x7fd2e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 0x7fd2e00a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00060b0 tx=0x7fd2f000b040 comp rx=0 tx=0).stop 2023-11-17T00:16:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 msgr2=0x7fd2f80ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800eb30 tx=0x7fd2e800ee40 comp rx=0 tx=0).stop 2023-11-17T00:16:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 shutdown_connections 2023-11-17T00:16:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd2e00a12c0 0x7fd2e00a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2f81052a0 0x7fd2f81a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2f8067dc0 0x7fd2f80ffb40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 --2- 172.21.15.6:0/463986671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd2f8068770 0x7fd2f8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 >> 172.21.15.6:0/463986671 conn(0x7fd2f80763a0 msgr2=0x7fd2f80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 shutdown_connections 2023-11-17T00:16:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:13.401+0000 7fd2fd1ff700 1 -- 172.21.15.6:0/463986671 wait complete. 2023-11-17T00:16:14.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:13 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/299305500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:14.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:13 smithi087 ceph-mon[131182]: osdmap e97: 12 total, 12 up, 12 in 2023-11-17T00:16:14.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:13 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/299305500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:14.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:13 smithi186 ceph-mon[131487]: osdmap e97: 12 total, 12 up, 12 in 2023-11-17T00:16:14.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:13 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/299305500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:14.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:13 smithi006 ceph-mon[97840]: osdmap e97: 12 total, 12 up, 12 in 2023-11-17T00:16:14.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:16:14.424 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:16:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1878045597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08105250 msgr2=0x7f7e08105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1878045597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08105250 0x7f7e08105620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009990 tx=0x7f7df8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1878045597 shutdown_connections 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1878045597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08068770 0x7f7e0810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1878045597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08105250 0x7f7e08105620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1878045597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08067dc0 0x7f7e08068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1878045597 >> 172.21.15.6:0/1878045597 conn(0x7f7e080763a0 msgr2=0x7f7e080767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1878045597 shutdown_connections 2023-11-17T00:16:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.638+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1878045597 wait complete. 2023-11-17T00:16:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.639+0000 7f7e0dbff700 1 Processor -- start 2023-11-17T00:16:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.639+0000 7f7e0dbff700 1 -- start start 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08068770 0x7f7e08198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08105250 0x7f7e0819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0819d630 con 0x7f7e08068770 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0819d7a0 con 0x7f7e08067dc0 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e0dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0819d960 con 0x7f7e08105250 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46444/0 (socket says 172.21.15.6:46444) 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 -- 172.21.15.6:0/1851335584 learned_addr learned my addr 172.21.15.6:0/1851335584 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08068770 0x7f7e08198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e07fff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08105250 0x7f7e0819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 -- 172.21.15.6:0/1851335584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08105250 msgr2=0x7f7e0819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08105250 0x7f7e0819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 -- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08068770 msgr2=0x7f7e08198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08068770 0x7f7e08198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.640+0000 7f7e077fe700 1 -- 172.21.15.6:0/1851335584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df80095f0 con 0x7f7e08067dc0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005910 tx=0x7f7df8004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e04ff9700 1 -- 172.21.15.6:0/1851335584 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df801d070 con 0x7f7e08067dc0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e04ff9700 1 -- 172.21.15.6:0/1851335584 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7df8022470 con 0x7f7e08067dc0 2023-11-17T00:16:14.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e08110400 con 0x7f7e08067dc0 2023-11-17T00:16:14.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e04ff9700 1 -- 172.21.15.6:0/1851335584 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8017650 con 0x7f7e08067dc0 2023-11-17T00:16:14.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.641+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08110270 con 0x7f7e08067dc0 2023-11-17T00:16:14.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.645+0000 7f7e04ff9700 1 -- 172.21.15.6:0/1851335584 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7df80225e0 con 0x7f7e08067dc0 2023-11-17T00:16:14.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.645+0000 7f7e21531b80 1 Processor -- start 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 -- start start 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x556656631fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x5566566363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656636d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566565370e0 con 0x5566566324f0 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656589920 con 0x556656631be0 2023-11-17T00:16:14.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e21531b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656582db0 con 0x556656636930 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x556656631fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x556656631fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46448/0 (socket says 172.21.15.6:46448) 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e07fff700 1 -- 172.21.15.6:0/300940528 learned_addr learned my addr 172.21.15.6:0/300940528 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e04ff9700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 0x7f7df00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e077fe700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x5566566363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.646+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656636d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e04ff9700 1 -- 172.21.15.6:0/1851335584 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7df80ce330 con 0x7f7e08067dc0 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 -- 172.21.15.6:0/300940528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 msgr2=0x556656631fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x556656631fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 -- 172.21.15.6:0/300940528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 msgr2=0x5566566363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x5566566363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 -- 172.21.15.6:0/300940528 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566564046b0 con 0x556656636930 2023-11-17T00:16:14.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e07fff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 0x7f7df00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e077fe700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x5566566363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:14.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656636d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009c80 tx=0x7f7dfc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3c955b7df2c8ae server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7dedffb700 1 -- 172.21.15.6:0/300940528 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc011cb0 con 0x556656636930 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.647+0000 7f7e07fff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 0x7f7df00a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009fd0 tx=0x7f7e00004b40 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7dedffb700 1 -- 172.21.15.6:0/300940528 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dfc011e10 con 0x556656636930 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7dedffb700 1 -- 172.21.15.6:0/300940528 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc005280 con 0x556656636930 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 -- 172.21.15.6:0/300940528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 msgr2=0x556656636d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656636d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009c80 tx=0x7f7dfc009f90 comp rx=0 tx=0).stop 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 -- 172.21.15.6:0/300940528 shutdown_connections 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656636d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x556656631fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 --2- 172.21.15.6:0/300940528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x5566566363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 -- 172.21.15.6:0/300940528 >> 172.21.15.6:0/300940528 conn(0x5566566273e0 msgr2=0x55665662cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 -- 172.21.15.6:0/300940528 shutdown_connections 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.648+0000 7f7e21531b80 1 -- 172.21.15.6:0/300940528 wait complete. 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.649+0000 7f7e21531b80 1 -- start start 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.649+0000 7f7e21531b80 1 Processor -- start 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 0x55665663b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656636930 0x55665663b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656524960 con 0x556656631be0 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656524600 con 0x556656636930 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566564069a0 con 0x5566566324f0 2023-11-17T00:16:14.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e21531b80 10 client.0 _ll_register_callbacks cb 0x556656542800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39364/0 (socket says 172.21.15.6:39364) 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e06ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 0x55665663b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 -- 172.21.15.6:0/4005868965 learned_addr learned my addr 172.21.15.6:0/4005868965 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e07fff700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656636930 0x55665663b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 -- 172.21.15.6:0/4005868965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 msgr2=0x55665663b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 0x55665663b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 -- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656636930 msgr2=0x55665663b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656636930 0x55665663b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e077fe700 1 -- 172.21.15.6:0/4005868965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc0098e0 con 0x556656631be0 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.650+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 0x55665663b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:14.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7e077fe700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7df809ad90 tx=0x7f7df8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b94f560329fad57 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7de6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df809a780 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7df809b090 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556656624ba0 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7df80ce150 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7e21531b80 10 client.16527 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556656549210 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.651+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7df80c9030 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7df80cb0b0 con 0x556656631be0 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 10 client.16527 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 20 client.16527 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665664b630 0x55665664da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 4 client.16527 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556656545120 con 0x55665664b630 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665664b630 0x55665664da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665664b630 0x55665664da10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282517dc95ef67b7 server_cookie=d943f994965fcce in_seq=0 out_seq=0 2023-11-17T00:16:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.652+0000 7f7de6ffd700 10 client.16527 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7de6ffd700 1 -- 172.21.15.6:0/4005868965 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556656545120 con 0x55665664b630 2023-11-17T00:16:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7de6ffd700 10 client.16527 handle_command_reply: tid=0 2023-11-17T00:16:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 client.16527 shutdown 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 msgr2=0x55665663e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7df809ad90 tx=0x7f7df8000bc0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 shutdown_connections 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665664b630 0x55665664da10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x55665663e0e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656636930 0x55665663b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 --2- 172.21.15.6:0/4005868965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566566324f0 0x55665663b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 >> 172.21.15.6:0/4005868965 conn(0x5566566273e0 msgr2=0x55665662be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7de6ffd700 0 client.16527 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:14.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 shutdown_connections 2023-11-17T00:16:14.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 1 -- 172.21.15.6:0/4005868965 wait complete. 2023-11-17T00:16:14.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.654+0000 7f7e21531b80 20 client.16527 trim_cache size 0 max 16384 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 Processor -- start 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 -- start start 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 0x556656642a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656637410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656626f60 con 0x556656631be0 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665662e240 con 0x5566566324f0 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e21531b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656639850 con 0x556656636930 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e06ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 0x556656642a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39368/0 (socket says 172.21.15.6:39368) 2023-11-17T00:16:14.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e077fe700 1 -- 172.21.15.6:0/1986565630 learned_addr learned my addr 172.21.15.6:0/1986565630 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.673+0000 7f7e07fff700 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656637410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 -- 172.21.15.6:0/1986565630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 msgr2=0x556656637410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656637410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 -- 172.21.15.6:0/1986565630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 msgr2=0x556656642a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 0x556656642a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 -- 172.21.15.6:0/1986565630 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556656548f60 con 0x556656631be0 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 0x556656642a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7e077fe700 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7df80993e0 tx=0x7f7df8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a45d955166958f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7dee7fc700 1 -- 172.21.15.6:0/1986565630 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80ce150 con 0x556656631be0 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7dee7fc700 1 -- 172.21.15.6:0/1986565630 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7df80c9030 con 0x556656631be0 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.674+0000 7f7dee7fc700 1 -- 172.21.15.6:0/1986565630 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8097240 con 0x556656631be0 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- 172.21.15.6:0/1986565630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 msgr2=0x556656638d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7df80993e0 tx=0x7f7df8005f10 comp rx=0 tx=0).stop 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- 172.21.15.6:0/1986565630 shutdown_connections 2023-11-17T00:16:14.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x556656631be0 0x556656638d40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656637410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 --2- 172.21.15.6:0/1986565630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5566566324f0 0x556656642a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- 172.21.15.6:0/1986565630 >> 172.21.15.6:0/1986565630 conn(0x5566566273e0 msgr2=0x556656636d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- 172.21.15.6:0/1986565630 shutdown_connections 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- 172.21.15.6:0/1986565630 wait complete. 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 -- start start 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.675+0000 7f7e21531b80 1 Processor -- start 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x55665663b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656639aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656639fe0 con 0x5566566324f0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665663a150 con 0x556656631be0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665663a310 con 0x556656636930 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e21531b80 10 client.0 _ll_register_callbacks cb 0x7f7df809abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e06ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x55665663b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46462/0 (socket says 172.21.15.6:46462) 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e077fe700 1 -- 172.21.15.6:0/381806891 learned_addr learned my addr 172.21.15.6:0/381806891 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.676+0000 7f7e07fff700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656639aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 -- 172.21.15.6:0/381806891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 msgr2=0x556656639aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656639aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 -- 172.21.15.6:0/381806891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 msgr2=0x55665663b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x55665663b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 -- 172.21.15.6:0/381806891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc01d040 con 0x556656631be0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e07fff700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656639aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e077fe700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7df80a2f20 tx=0x7f7df8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d47c907729fb4d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7de6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80ce150 con 0x556656631be0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dfc011e10 con 0x556656631be0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7df80a3450 con 0x556656631be0 2023-11-17T00:16:14.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7df809b090 con 0x556656631be0 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e21531b80 10 client.26672 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.677+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556656549210 con 0x556656631be0 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dfc011e10 con 0x556656631be0 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7df8099ff0 con 0x556656631be0 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 10 client.26672 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 20 client.26672 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665669fe10 0x5566566a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 4 client.26672 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5566565516e0 con 0x55665669fe10 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.678+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665669fe10 0x5566566a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.679+0000 7f7de6ffd700 10 client.26672 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:14.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.679+0000 7f7e06ffd700 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665669fe10 0x5566566a21f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9c9abda874b665 server_cookie=ae623714aa96297d in_seq=0 out_seq=0 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.680+0000 7f7de6ffd700 1 -- 172.21.15.6:0/381806891 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5566565516e0 con 0x55665669fe10 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.680+0000 7f7de6ffd700 10 client.26672 handle_command_reply: tid=0 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 client.26672 shutdown 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 msgr2=0x55665663ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7df80a2f20 tx=0x7f7df8009310 comp rx=0 tx=0).stop 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 shutdown_connections 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556656636930 0x556656639aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556656631be0 0x55665663ac70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55665669fe10 0x5566566a21f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 --2- 172.21.15.6:0/381806891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5566566324f0 0x55665663b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 >> 172.21.15.6:0/381806891 conn(0x5566566273e0 msgr2=0x556656635830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7de6ffd700 0 client.26672 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:14.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 shutdown_connections 2023-11-17T00:16:14.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 1 -- 172.21.15.6:0/381806891 wait complete. 2023-11-17T00:16:14.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.681+0000 7f7e21531b80 20 client.26672 trim_cache size 0 max 16384 2023-11-17T00:16:14.692 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:16:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "4e110d88-2011-448c-bf8e-79a08982359f", 2023-11-17T00:16:14.692 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:16:14.692 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:14.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 msgr2=0x7f7df00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 0x7f7df00a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009fd0 tx=0x7f7e00004b40 comp rx=0 tx=0).stop 2023-11-17T00:16:14.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 msgr2=0x7f7e08198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005910 tx=0x7f7df8004710 comp rx=0 tx=0).stop 2023-11-17T00:16:14.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 shutdown_connections 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7df00a1340 0x7f7df00a37f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e08105250 0x7f7e0819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e08067dc0 0x7f7e08198650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 --2- 172.21.15.6:0/1851335584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e08068770 0x7f7e08198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 >> 172.21.15.6:0/1851335584 conn(0x7f7e080763a0 msgr2=0x7f7e0810b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 shutdown_connections 2023-11-17T00:16:14.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.682+0000 7f7e0dbff700 1 -- 172.21.15.6:0/1851335584 wait complete. 2023-11-17T00:16:14.708 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 -- 172.21.15.6:0/2245810854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc90105560 msgr2=0x7fdc9010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 --2- 172.21.15.6:0/2245810854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc90105560 0x7fdc9010d130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009fc0 tx=0x7fdc8800b330 comp rx=0 tx=0).stop 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 -- 172.21.15.6:0/2245810854 shutdown_connections 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 --2- 172.21.15.6:0/2245810854 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc90105560 0x7fdc9010d130 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 --2- 172.21.15.6:0/2245810854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90100c50 0x7fdc90101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 --2- 172.21.15.6:0/2245810854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc901015f0 0x7fdc90104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 -- 172.21.15.6:0/2245810854 >> 172.21.15.6:0/2245810854 conn(0x7fdc90077500 msgr2=0x7fdc90077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 -- 172.21.15.6:0/2245810854 shutdown_connections 2023-11-17T00:16:14.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.916+0000 7fdc95bff700 1 -- 172.21.15.6:0/2245810854 wait complete. 2023-11-17T00:16:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 Processor -- start 2023-11-17T00:16:14.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 -- start start 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc90100c50 0x7fdc9019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 0x7fdc901985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90198c10 con 0x7fdc901015f0 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90198d80 con 0x7fdc90105560 2023-11-17T00:16:14.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.917+0000 7fdc95bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90198f40 con 0x7fdc90100c50 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39400/0 (socket says 172.21.15.6:39400) 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 0x7fdc901985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8effd700 1 -- 172.21.15.6:0/4087416099 learned_addr learned my addr 172.21.15.6:0/4087416099 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc90100c50 0x7fdc9019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 -- 172.21.15.6:0/4087416099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc90100c50 msgr2=0x7fdc9019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc90100c50 0x7fdc9019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 -- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 msgr2=0x7fdc9019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 -- 172.21.15.6:0/4087416099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc88009c40 con 0x7fdc90105560 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8effd700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.918+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 0x7fdc901985a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc88015000 tx=0x7fdc88009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.919+0000 7fdc8cff9700 1 -- 172.21.15.6:0/4087416099 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800e040 con 0x7fdc90105560 2023-11-17T00:16:14.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.919+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc90110360 con 0x7fdc90105560 2023-11-17T00:16:14.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.919+0000 7fdc8cff9700 1 -- 172.21.15.6:0/4087416099 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8800bbf0 con 0x7fdc90105560 2023-11-17T00:16:14.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.919+0000 7fdc8cff9700 1 -- 172.21.15.6:0/4087416099 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880052a0 con 0x7fdc90105560 2023-11-17T00:16:14.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.919+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc901101d0 con 0x7fdc90105560 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.922+0000 7fdc8cff9700 1 -- 172.21.15.6:0/4087416099 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdc8801d470 con 0x7fdc90105560 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.923+0000 7fdca943cb80 1 Processor -- start 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.923+0000 7fdc8cff9700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 0x7fdc780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.923+0000 7fdca943cb80 1 -- start start 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8cff9700 1 -- 172.21.15.6:0/4087416099 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdc880ce5f0 con 0x7fdc90105560 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8effd700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 0x7fdc780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96311df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963122370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963023060 con 0x55b96311e470 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9630758a0 con 0x55b9631228b0 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdca943cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96306ed30 con 0x55b96311db60 2023-11-17T00:16:14.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46484/0 (socket says 172.21.15.6:46484) 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8ffff700 1 -- 172.21.15.6:0/2626496971 learned_addr learned my addr 172.21.15.6:0/2626496971 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8effd700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963122370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.924+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96311df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 -- 172.21.15.6:0/2626496971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 msgr2=0x55b96311df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96311df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 -- 172.21.15.6:0/2626496971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 msgr2=0x55b963122370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963122370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 -- 172.21.15.6:0/2626496971 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b962ef0630 con 0x55b9631228b0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8effd700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963122370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8effd700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 0x7fdc780a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc90106850 tx=0x7fdc84010760 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc88097dc0 tx=0x7fdc88009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a99ae22448808591 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc75ffb700 1 -- 172.21.15.6:0/2626496971 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880ce020 con 0x55b9631228b0 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc75ffb700 1 -- 172.21.15.6:0/2626496971 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8809acb0 con 0x55b9631228b0 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.925+0000 7fdc75ffb700 1 -- 172.21.15.6:0/2626496971 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880a10d0 con 0x55b9631228b0 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 -- 172.21.15.6:0/2626496971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 msgr2=0x55b963122d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc88097dc0 tx=0x7fdc88009bb0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 -- 172.21.15.6:0/2626496971 shutdown_connections 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963122d00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96311df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 --2- 172.21.15.6:0/2626496971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963122370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 -- 172.21.15.6:0/2626496971 >> 172.21.15.6:0/2626496971 conn(0x55b963113360 msgr2=0x55b963118b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 -- 172.21.15.6:0/2626496971 shutdown_connections 2023-11-17T00:16:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.926+0000 7fdca943cb80 1 -- 172.21.15.6:0/2626496971 wait complete. 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.927+0000 7fdca943cb80 1 -- start start 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.927+0000 7fdca943cb80 1 Processor -- start 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96312a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963127210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9630108e0 con 0x55b96311e470 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963010580 con 0x55b9631228b0 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b962ef2920 con 0x55b96311db60 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdca943cb80 10 client.0 _ll_register_callbacks cb 0x55b96302e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963127210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96312a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46496/0 (socket says 172.21.15.6:46496) 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 -- 172.21.15.6:0/3136787761 learned_addr learned my addr 172.21.15.6:0/3136787761 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 -- 172.21.15.6:0/3136787761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 msgr2=0x55b96312a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96312a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 -- 172.21.15.6:0/3136787761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 msgr2=0x55b963127210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963127210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.928+0000 7fdc8ffff700 1 -- 172.21.15.6:0/3136787761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc901101d0 con 0x55b9631228b0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc8809b050 tx=0x7fdc8800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a9b5daef206be3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880ce040 con 0x55b9631228b0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdc8809f380 con 0x55b9631228b0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b963110b20 con 0x55b9631228b0 2023-11-17T00:16:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc880cbdd0 con 0x55b9631228b0 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdca943cb80 10 client.26684 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b963035190 con 0x55b9631228b0 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b963110b20 con 0x55b9631228b0 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc8809b5b0 con 0x55b9631228b0 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdca943cb80 10 client.26684 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.929+0000 7fdca943cb80 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdca943cb80 20 client.26684 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b963137140 0x55b963139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdca943cb80 4 client.26684 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9630310a0 con 0x55b963137140 2023-11-17T00:16:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b963137140 0x55b963139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:14.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b963137140 0x55b963139550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8896ea52b5b76d2 server_cookie=648c8efafdcd74f0 in_seq=0 out_seq=0 2023-11-17T00:16:14.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:14.930+0000 7fdc6effd700 10 client.26684 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:15.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:14 smithi087 ceph-mon[131182]: pgmap v226: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:15.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.108+0000 7fdc6effd700 1 -- 172.21.15.6:0/3136787761 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b9630310a0 con 0x55b963137140 2023-11-17T00:16:15.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.108+0000 7fdc6effd700 10 client.26684 handle_command_reply: tid=0 2023-11-17T00:16:15.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.108+0000 7fdca943cb80 1 client.26684 shutdown 2023-11-17T00:16:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 msgr2=0x55b963127800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc8809b050 tx=0x7fdc8800bf90 comp rx=0 tx=0).stop 2023-11-17T00:16:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 shutdown_connections 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b9631228b0 0x55b963127800 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b96311db60 0x55b96312a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b963137140 0x55b963139550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 --2- 172.21.15.6:0/3136787761 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b963127210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdc6effd700 0 client.26684 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 >> 172.21.15.6:0/3136787761 conn(0x55b963113360 msgr2=0x55b963123d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 shutdown_connections 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 1 -- 172.21.15.6:0/3136787761 wait complete. 2023-11-17T00:16:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.109+0000 7fdca943cb80 20 client.26684 trim_cache size 0 max 16384 2023-11-17T00:16:15.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:14 smithi186 ceph-mon[131487]: pgmap v226: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 Processor -- start 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 -- start start 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 0x55b963129b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b963117f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96311ebc0 con 0x55b96311db60 2023-11-17T00:16:15.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963119780 con 0x55b96311e470 2023-11-17T00:16:15.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.128+0000 7fdca943cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b963127e00 con 0x55b9631228b0 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46510/0 (socket says 172.21.15.6:46510) 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 -- 172.21.15.6:0/1017396259 learned_addr learned my addr 172.21.15.6:0/1017396259 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 0x55b963129b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b963117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 -- 172.21.15.6:0/1017396259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 msgr2=0x55b963117f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:15.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b963117f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 -- 172.21.15.6:0/1017396259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 msgr2=0x55b963129b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 0x55b963129b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 -- 172.21.15.6:0/1017396259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b963126dd0 con 0x55b96311e470 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 0x55b963129b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.129+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc840137a0 tx=0x7fdc84015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169a45c57a4c946d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.130+0000 7fdc767fc700 1 -- 172.21.15.6:0/1017396259 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84024070 con 0x55b96311e470 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.130+0000 7fdc767fc700 1 -- 172.21.15.6:0/1017396259 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc840044e0 con 0x55b96311e470 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.130+0000 7fdc767fc700 1 -- 172.21.15.6:0/1017396259 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84003e90 con 0x55b96311e470 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- 172.21.15.6:0/1017396259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 msgr2=0x55b9631179f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc840137a0 tx=0x7fdc84015000 comp rx=0 tx=0).stop 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- 172.21.15.6:0/1017396259 shutdown_connections 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311db60 0x55b963129b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b963117f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1017396259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311e470 0x55b9631179f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- 172.21.15.6:0/1017396259 >> 172.21.15.6:0/1017396259 conn(0x55b963113360 msgr2=0x55b9631208b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- 172.21.15.6:0/1017396259 shutdown_connections 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- 172.21.15.6:0/1017396259 wait complete. 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 -- start start 2023-11-17T00:16:15.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.131+0000 7fdca943cb80 1 Processor -- start 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b96312c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b96312cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96312d300 con 0x55b96311e470 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96312d470 con 0x55b96311db60 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b96312d630 con 0x55b9631228b0 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdca943cb80 10 client.0 _ll_register_callbacks cb 0x7fdc80009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:15.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdc8f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdc8f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46526/0 (socket says 172.21.15.6:46526) 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdc8f7fe700 1 -- 172.21.15.6:0/1177002594 learned_addr learned my addr 172.21.15.6:0/1177002594 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b96312c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.132+0000 7fdc8ffff700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b96312cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 -- 172.21.15.6:0/1177002594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 msgr2=0x55b96312cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b96312cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 -- 172.21.15.6:0/1177002594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 msgr2=0x55b96312c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b96312c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 -- 172.21.15.6:0/1177002594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80009c50 con 0x55b96311db60 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b96312c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:15.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc8f7fe700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b963139e30 tx=0x7fdc80010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408884400324434b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc6effd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80017040 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc8809acb0 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdc80010e10 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8001f610 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdca943cb80 10 client.26690 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.133+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fdc84013480 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8809acb0 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc8001e440 con 0x55b96311db60 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 10 client.26690 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:15.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 20 client.26690 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b96318c440 0x55b96318e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 4 client.26690 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b96303d660 con 0x55b96318c440 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.134+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b96318c440 0x55b96318e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.135+0000 7fdc8effd700 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b96318c440 0x55b96318e850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178813cfef7983ea server_cookie=fe67bad41e9a9605 in_seq=0 out_seq=0 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.135+0000 7fdc6effd700 10 client.26690 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.135+0000 7fdc6effd700 1 -- 172.21.15.6:0/1177002594 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b96303d660 con 0x55b96318c440 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdc6effd700 10 client.26690 handle_command_reply: tid=0 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 client.26690 shutdown 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 msgr2=0x55b96312e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b963139e30 tx=0x7fdc80010710 comp rx=0 tx=0).stop 2023-11-17T00:16:15.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 shutdown_connections 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b9631228b0 0x55b96312cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b96311db60 0x55b96312e550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b96318c440 0x55b96318e850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 --2- 172.21.15.6:0/1177002594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b96311e470 0x55b96312c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdc6effd700 0 client.26690 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 >> 172.21.15.6:0/1177002594 conn(0x55b963113360 msgr2=0x55b9631208b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 shutdown_connections 2023-11-17T00:16:15.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 1 -- 172.21.15.6:0/1177002594 wait complete. 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.136+0000 7fdca943cb80 20 client.26690 trim_cache size 0 max 16384 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "4e110d88-2011-448c-bf8e-79a08982359f": { 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "4e110d88-2011-448c-bf8e-79a08982359f", 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:16:15.142 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:16:15.143 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:15.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 msgr2=0x7fdc780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:15.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 0x7fdc780a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc90106850 tx=0x7fdc84010760 comp rx=0 tx=0).stop 2023-11-17T00:16:15.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 msgr2=0x7fdc901985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:15.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 0x7fdc901985a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc88015000 tx=0x7fdc88009480 comp rx=0 tx=0).stop 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 shutdown_connections 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc780a12c0 0x7fdc780a3770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc90105560 0x7fdc901985a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc90100c50 0x7fdc9019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 --2- 172.21.15.6:0/4087416099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc901015f0 0x7fdc9019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 >> 172.21.15.6:0/4087416099 conn(0x7fdc90077500 msgr2=0x7fdc900fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 shutdown_connections 2023-11-17T00:16:15.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:15.137+0000 7fdc95bff700 1 -- 172.21.15.6:0/4087416099 wait complete. 2023-11-17T00:16:15.157 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:14 smithi006 ceph-mon[97840]: pgmap v226: 129 pgs: 129 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.03k op/s 2023-11-17T00:16:15.158 INFO:tasks.cephfs.filesystem:scrub status for tag:4e110d88-2011-448c-bf8e-79a08982359f - {'path': '/', 'tag': '4e110d88-2011-448c-bf8e-79a08982359f', 'options': 'recursive,force'} 2023-11-17T00:16:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:15 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:16:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:15 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:15 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:16:16.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:15 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:15 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:16:16.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:15 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:15 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:16:16.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:15 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:15 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:16:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:15 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:15 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:16:16.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:15 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:16.265 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:16:16.266 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:16:16.344 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-17T00:16:16.344 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-17T00:16:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:16 smithi087 ceph-mon[131182]: pgmap v228: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.13k op/s 2023-11-17T00:16:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:16 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:16:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:16 smithi186 ceph-mon[131487]: pgmap v228: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.13k op/s 2023-11-17T00:16:17.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:16 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:16:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:16 smithi006 ceph-mon[97840]: pgmap v228: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.13k op/s 2023-11-17T00:16:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:16 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:16:19.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:18 smithi087 ceph-mon[131182]: pgmap v229: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:19.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:18 smithi186 ceph-mon[131487]: pgmap v229: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:19.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:18 smithi006 ceph-mon[97840]: pgmap v229: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:20 smithi087 ceph-mon[131182]: pgmap v230: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:20 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:16:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:20 smithi186 ceph-mon[131487]: pgmap v230: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:21.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:20 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:16:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:20 smithi006 ceph-mon[97840]: pgmap v230: 129 pgs: 129 active+clean; 8.0 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 1.08k op/s 2023-11-17T00:16:21.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:20 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:16:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:22 smithi087 ceph-mon[131182]: pgmap v231: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:22 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:16:23.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:22 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:16:23.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:22 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:16:23.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:22 smithi186 ceph-mon[131487]: pgmap v231: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:23.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:22 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:16:23.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:22 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:16:23.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:22 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:16:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:22 smithi006 ceph-mon[97840]: pgmap v231: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:22 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:16:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:22 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:16:23.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:22 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:16:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:23 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:16:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:23 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:16:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:23 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:16:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:23 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:16:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:23 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:16:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:23 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:16:24.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:23 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:16:24.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:23 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:16:24.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:23 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:16:24.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:23 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:16:24.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:23 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:16:24.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:23 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:16:25.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:24 smithi087 ceph-mon[131182]: pgmap v232: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:24 smithi186 ceph-mon[131487]: pgmap v232: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:25.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:24 smithi006 ceph-mon[97840]: pgmap v232: 129 pgs: 129 active+clean; 8.1 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 804 KiB/s wr, 621 op/s 2023-11-17T00:16:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 -- 172.21.15.6:0/241060322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 msgr2=0x7f8440110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 --2- 172.21.15.6:0/241060322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f8440110d00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8434009990 tx=0x7f8434009ca0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 -- 172.21.15.6:0/241060322 shutdown_connections 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 --2- 172.21.15.6:0/241060322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 0x7f8440113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 --2- 172.21.15.6:0/241060322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 0x7f8440073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 --2- 172.21.15.6:0/241060322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f8440110d00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 -- 172.21.15.6:0/241060322 >> 172.21.15.6:0/241060322 conn(0x7f84400fbd10 msgr2=0x7f84400fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 -- 172.21.15.6:0/241060322 shutdown_connections 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.685+0000 7f84467ff700 1 -- 172.21.15.6:0/241060322 wait complete. 2023-11-17T00:16:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.687+0000 7f84467ff700 1 Processor -- start 2023-11-17T00:16:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.687+0000 7f84467ff700 1 -- start start 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.687+0000 7f84467ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 0x7f84401a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.687+0000 7f84467ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.687+0000 7f84467ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 0x7f844019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f84467ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844019ce70 con 0x7f84400737a0 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f84467ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844019cfe0 con 0x7f8440111240 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f84467ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844019d1a0 con 0x7f8440072e90 2023-11-17T00:16:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43124/0 (socket says 172.21.15.6:43124) 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 -- 172.21.15.6:0/1063480821 learned_addr learned my addr 172.21.15.6:0/1063480821 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843ffff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 0x7f84401a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f8444d9c700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 0x7f844019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 -- 172.21.15.6:0/1063480821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 msgr2=0x7f84401a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 0x7f84401a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 -- 172.21.15.6:0/1063480821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 msgr2=0x7f844019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 0x7f844019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 -- 172.21.15.6:0/1063480821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340095f0 con 0x7f84400737a0 2023-11-17T00:16:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.688+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8434005910 tx=0x7f8434005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.689+0000 7f843d7fa700 1 -- 172.21.15.6:0/1063480821 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843401d070 con 0x7f84400737a0 2023-11-17T00:16:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.689+0000 7f843d7fa700 1 -- 172.21.15.6:0/1063480821 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8434022470 con 0x7f84400737a0 2023-11-17T00:16:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.689+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84401168d0 con 0x7f84400737a0 2023-11-17T00:16:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.689+0000 7f843d7fa700 1 -- 172.21.15.6:0/1063480821 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434017650 con 0x7f84400737a0 2023-11-17T00:16:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.689+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440116740 con 0x7f84400737a0 2023-11-17T00:16:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.692+0000 7f843d7fa700 1 -- 172.21.15.6:0/1063480821 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f84340225e0 con 0x7f84400737a0 2023-11-17T00:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.693+0000 7f843d7fa700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 0x7f84280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.693+0000 7f845a1e5b80 1 Processor -- start 2023-11-17T00:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f843d7fa700 1 -- 172.21.15.6:0/1063480821 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f84340ce370 con 0x7f84400737a0 2023-11-17T00:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 -- start start 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f843ffff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 0x7f84280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff180491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18043b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17f49000 con 0x55ff18049b90 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17f9b840 con 0x55ff17e9f6b0 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.694+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17f94cd0 con 0x55ff18049720 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843ffff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 0x7f84280a37f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8430009c60 tx=0x7f8430009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:59856/0 (socket says 172.21.15.6:59856) 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 -- 172.21.15.6:0/2826843093 learned_addr learned my addr 172.21.15.6:0/2826843093 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f8444d9c700 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18043b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843ffff700 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff180491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 -- 172.21.15.6:0/2826843093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 msgr2=0x55ff180491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff180491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 -- 172.21.15.6:0/2826843093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 msgr2=0x55ff18043b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18043b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 -- 172.21.15.6:0/2826843093 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff17e165d0 con 0x55ff18049720 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84340975d0 tx=0x7f8434005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7eeff8ad7dc76fb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f84267fc700 1 -- 172.21.15.6:0/2826843093 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843409aa30 con 0x55ff18049720 2023-11-17T00:16:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.695+0000 7f84267fc700 1 -- 172.21.15.6:0/2826843093 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843409b0d0 con 0x55ff18049720 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f84267fc700 1 -- 172.21.15.6:0/2826843093 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843409b330 con 0x55ff18049720 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2826843093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 msgr2=0x55ff18043560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84340975d0 tx=0x7f8434005f10 comp rx=0 tx=0).stop 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2826843093 shutdown_connections 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18043b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff180491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2826843093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049720 0x55ff18043560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2826843093 >> 172.21.15.6:0/2826843093 conn(0x55ff18038f50 msgr2=0x55ff1803e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2826843093 shutdown_connections 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.696+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2826843093 wait complete. 2023-11-17T00:16:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.697+0000 7f845a1e5b80 1 -- start start 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.697+0000 7f845a1e5b80 1 Processor -- start 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18052cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff1804ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17f36880 con 0x55ff17e9f6b0 2023-11-17T00:16:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17f36520 con 0x55ff18049720 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff17e188c0 con 0x55ff18049b90 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f845a1e5b80 10 client.0 _ll_register_callbacks cb 0x55ff17f54720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43132/0 (socket says 172.21.15.6:43132) 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 -- 172.21.15.6:0/1700446332 learned_addr learned my addr 172.21.15.6:0/1700446332 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f8444d9c700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff1804ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18052cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 -- 172.21.15.6:0/1700446332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 msgr2=0x55ff1804ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff1804ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 -- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 msgr2=0x55ff18052cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18052cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.698+0000 7f843ffff700 1 -- 172.21.15.6:0/1700446332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440116740 con 0x55ff17e9f6b0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f843ffff700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8430003930 tx=0x7f84300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2667b8082f8191 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f841f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430022070 con 0x55ff17e9f6b0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff18036710 con 0x55ff17e9f6b0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f843001d620 con 0x55ff17e9f6b0 2023-11-17T00:16:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8430004830 con 0x55ff17e9f6b0 2023-11-17T00:16:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f845a1e5b80 10 client.16563 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ff17f5b130 con 0x55ff17e9f6b0 2023-11-17T00:16:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.699+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8430030020 con 0x55ff17e9f6b0 2023-11-17T00:16:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8430005020 con 0x55ff17e9f6b0 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 10 client.16563 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 20 client.16563 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff1805cd10 0x55ff1805f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 4 client.16563 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff17f57040 con 0x55ff1805cd10 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.700+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff1805cd10 0x55ff1805f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.701+0000 7f843f7fe700 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff1805cd10 0x55ff1805f120 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1972443ad41c95d server_cookie=4a82a46663e54c28 in_seq=0 out_seq=0 2023-11-17T00:16:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.701+0000 7f841f7fe700 10 client.16563 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f841f7fe700 1 -- 172.21.15.6:0/1700446332 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff17f57040 con 0x55ff1805cd10 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f841f7fe700 10 client.16563 handle_command_reply: tid=0 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 client.16563 shutdown 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 msgr2=0x55ff180527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8430003930 tx=0x7f84300047c0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 shutdown_connections 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff1805cd10 0x55ff1805f120 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff17e9f6b0 0x55ff180527b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff1804ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f841f7fe700 0 client.16563 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/1700446332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18052cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 >> 172.21.15.6:0/1700446332 conn(0x55ff18038f50 msgr2=0x55ff1803d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 shutdown_connections 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.702+0000 7f845a1e5b80 1 -- 172.21.15.6:0/1700446332 wait complete. 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.703+0000 7f845a1e5b80 20 client.16563 trim_cache size 0 max 16384 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 Processor -- start 2023-11-17T00:16:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 -- start start 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff1804c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff18045c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.718+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1803afd0 con 0x55ff18049720 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1804dfe0 con 0x55ff17e9f6b0 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1803efb0 con 0x55ff18049b90 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43134/0 (socket says 172.21.15.6:43134) 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 -- 172.21.15.6:0/2121172528 learned_addr learned my addr 172.21.15.6:0/2121172528 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843ffff700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff1804c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f8444d9c700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff18045c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 -- 172.21.15.6:0/2121172528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 msgr2=0x55ff18045c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff18045c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 -- 172.21.15.6:0/2121172528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 msgr2=0x55ff1804c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff1804c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 -- 172.21.15.6:0/2121172528 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff1803a6f0 con 0x55ff18049720 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843ffff700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff1804c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.719+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8434097b40 tx=0x7f8434022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588cd6c1aa9ec8c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f8426ffd700 1 -- 172.21.15.6:0/2121172528 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340a3490 con 0x55ff18049720 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f8426ffd700 1 -- 172.21.15.6:0/2121172528 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843409f100 con 0x55ff18049720 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f8426ffd700 1 -- 172.21.15.6:0/2121172528 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340cb530 con 0x55ff18049720 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2121172528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 msgr2=0x55ff1803df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8434097b40 tx=0x7f8434022c90 comp rx=0 tx=0).stop 2023-11-17T00:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2121172528 shutdown_connections 2023-11-17T00:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff18049b90 0x55ff18045c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff17e9f6b0 0x55ff1804c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2121172528 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049720 0x55ff1803df60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2121172528 >> 172.21.15.6:0/2121172528 conn(0x55ff18038f50 msgr2=0x55ff1804c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2121172528 shutdown_connections 2023-11-17T00:16:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.720+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2121172528 wait complete. 2023-11-17T00:16:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.721+0000 7f845a1e5b80 1 -- start start 2023-11-17T00:16:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.721+0000 7f845a1e5b80 1 Processor -- start 2023-11-17T00:16:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 0x55ff18052c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18050320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff18051150 con 0x55ff18049b90 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff180afbd0 con 0x55ff18049720 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff180afd10 con 0x55ff17e9f6b0 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f845a1e5b80 10 client.0 _ll_register_callbacks cb 0x55ff17f54720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:43148/0 (socket says 172.21.15.6:43148) 2023-11-17T00:16:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 -- 172.21.15.6:0/2542356876 learned_addr learned my addr 172.21.15.6:0/2542356876 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f843f7fe700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18050320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f843ffff700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 0x55ff18052c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 -- 172.21.15.6:0/2542356876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 msgr2=0x55ff18052c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 0x55ff18052c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 -- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 msgr2=0x55ff18050320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18050320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f8444d9c700 1 -- 172.21.15.6:0/2542356876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff17f5b130 con 0x55ff18049b90 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.722+0000 7f843ffff700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 0x55ff18052c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f8444d9c700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8438011cb0 tx=0x7f8438011fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f8e4b5b4fbaa4a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843801c070 con 0x55ff18049b90 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f843409b0d0 con 0x55ff18049b90 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8438007cd0 con 0x55ff18049b90 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f843800e040 con 0x55ff18049b90 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f845a1e5b80 10 client.16569 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ff17e165d0 con 0x55ff18049b90 2023-11-17T00:16:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843409b0d0 con 0x55ff18049b90 2023-11-17T00:16:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.723+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8438021ce0 con 0x55ff18049b90 2023-11-17T00:16:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 10 client.16569 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 20 client.16569 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff180b1490 0x55ff180b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 4 client.16569 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:16:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff17f63600 con 0x55ff180b1490 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.724+0000 7f843ffff700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff180b1490 0x55ff180b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.725+0000 7f843ffff700 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff180b1490 0x55ff180b38a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7f71d8801d39c server_cookie=9b9108b92f65dd3d in_seq=0 out_seq=0 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.725+0000 7f841f7fe700 10 client.16569 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f841f7fe700 1 -- 172.21.15.6:0/2542356876 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff17f63600 con 0x55ff180b1490 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f841f7fe700 10 client.16569 handle_command_reply: tid=0 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 client.16569 shutdown 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 msgr2=0x55ff18050a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8438011cb0 tx=0x7f8438011fc0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 shutdown_connections 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ff180b1490 0x55ff180b38a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ff18049b90 0x55ff18050a10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ff17e9f6b0 0x55ff18052c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 --2- 172.21.15.6:0/2542356876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ff18049720 0x55ff18050320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f841f7fe700 0 client.16569 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 >> 172.21.15.6:0/2542356876 conn(0x55ff18038f50 msgr2=0x55ff1804b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 shutdown_connections 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.960+0000 7f845a1e5b80 1 -- 172.21.15.6:0/2542356876 wait complete. 2023-11-17T00:16:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.961+0000 7f845a1e5b80 20 client.16569 trim_cache size 0 max 16384 2023-11-17T00:16:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:16:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:16:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:16:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:16:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.961+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 msgr2=0x7f84280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.961+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 0x7f84280a37f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8430009c60 tx=0x7f8430009260 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.961+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 msgr2=0x7f84401a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.961+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8434005910 tx=0x7f8434005720 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 shutdown_connections 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f84280a1340 0x7f84280a37f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8440111240 0x7f844019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8440072e90 0x7f84401a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 --2- 172.21.15.6:0/1063480821 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84400737a0 0x7f84401a2cd0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 >> 172.21.15.6:0/1063480821 conn(0x7f84400fbd10 msgr2=0x7f84400fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 shutdown_connections 2023-11-17T00:16:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:25.962+0000 7f84467ff700 1 -- 172.21.15.6:0/1063480821 wait complete. 2023-11-17T00:16:27.315 INFO:tasks.workunit.client.0.smithi006.stdout:ok 2023-11-17T00:16:27.316 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:16:27.316 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:16:27.316 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:26 smithi006 ceph-mon[97840]: pgmap v233: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 718 KiB/s wr, 523 op/s 2023-11-17T00:16:27.317 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:26 smithi186 ceph-mon[131487]: pgmap v233: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 718 KiB/s wr, 523 op/s 2023-11-17T00:16:27.317 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:26 smithi087 ceph-mon[131182]: pgmap v233: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 718 KiB/s wr, 523 op/s 2023-11-17T00:16:27.413 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-17T00:16:27.414 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-17T00:16:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:28 smithi087 ceph-mon[131182]: pgmap v234: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 106 KiB/s wr, 122 op/s 2023-11-17T00:16:29.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:28 smithi186 ceph-mon[131487]: pgmap v234: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 106 KiB/s wr, 122 op/s 2023-11-17T00:16:29.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:28 smithi006 ceph-mon[97840]: pgmap v234: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 106 KiB/s wr, 122 op/s 2023-11-17T00:16:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:30 smithi087 ceph-mon[131182]: pgmap v235: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 44 op/s 2023-11-17T00:16:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:30 smithi186 ceph-mon[131487]: pgmap v235: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 44 op/s 2023-11-17T00:16:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:30 smithi006 ceph-mon[97840]: pgmap v235: 129 pgs: 129 active+clean; 8.6 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 58 KiB/s wr, 44 op/s 2023-11-17T00:16:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:32 smithi186 ceph-mon[131487]: pgmap v236: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 88 KiB/s wr, 210 op/s 2023-11-17T00:16:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:32 smithi006 ceph-mon[97840]: pgmap v236: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 88 KiB/s wr, 210 op/s 2023-11-17T00:16:33.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:32 smithi087 ceph-mon[131182]: pgmap v236: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 88 KiB/s wr, 210 op/s 2023-11-17T00:16:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:34 smithi186 ceph-mon[131487]: pgmap v237: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 189 op/s 2023-11-17T00:16:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:34 smithi006 ceph-mon[97840]: pgmap v237: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 189 op/s 2023-11-17T00:16:35.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:34 smithi087 ceph-mon[131182]: pgmap v237: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 189 op/s 2023-11-17T00:16:37.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:36 smithi186 ceph-mon[131487]: pgmap v238: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 191 op/s 2023-11-17T00:16:37.227 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:36 smithi006 ceph-mon[97840]: pgmap v238: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 191 op/s 2023-11-17T00:16:37.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:36 smithi087 ceph-mon[131182]: pgmap v238: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 191 op/s 2023-11-17T00:16:37.508 INFO:tasks.workunit.client.0.smithi006.stdout:ok 2023-11-17T00:16:37.513 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-17T00:16:37.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-17T00:16:37.603 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-17T00:16:37.603 DEBUG:teuthology.orchestra.run.smithi006:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-17T00:16:37.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir -p dir 2023-11-17T00:16:37.693 INFO:tasks.workunit.client.0.smithi006.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-17T00:16:37.694 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-17T00:16:37.696 INFO:tasks.workunit.client.0.smithi006.stdout:ceph.dir.pin="-1" 2023-11-17T00:16:37.697 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-17T00:16:37.697 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-17T00:16:37.699 INFO:tasks.workunit.client.0.smithi006.stdout:setfattr: dir: Invalid argument 2023-11-17T00:16:37.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-17T00:16:37.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-17T00:16:37.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-17T00:16:37.706 INFO:tasks.workunit.client.0.smithi006.stdout:ceph.dir.pin="1" 2023-11-17T00:16:37.706 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-17T00:16:37.710 INFO:tasks.workunit.client.0.smithi006.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-17T00:16:37.710 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-17T00:16:37.712 INFO:tasks.workunit.client.0.smithi006.stdout:ceph.dir.pin="-1" 2023-11-17T00:16:37.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-17T00:16:37.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-17T00:16:37.719 INFO:tasks.workunit.client.0.smithi006.stdout:setfattr: dir: Invalid argument 2023-11-17T00:16:37.721 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-17T00:16:37.721 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-17T00:16:37.727 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-17T00:16:37.728 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-17T00:16:37.730 INFO:tasks.workunit.client.0.smithi006.stdout:setfattr: dir: Invalid argument 2023-11-17T00:16:37.730 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-17T00:16:37.730 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-17T00:16:37.740 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-17T00:16:37.740 INFO:tasks.workunit.client.0.smithi006.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-17T00:16:37.741 INFO:tasks.workunit.client.0.smithi006.stdout:setfattr: dir: Invalid argument 2023-11-17T00:16:37.741 INFO:tasks.workunit.client.0.smithi006.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-17T00:16:37.743 INFO:tasks.workunit.client.0.smithi006.stderr:setfattr: dir: Invalid argument 2023-11-17T00:16:37.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:16:37.745 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-17T00:16:37.745 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 -- 172.21.15.6:0/368650496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 msgr2=0x7f88380fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 --2- 172.21.15.6:0/368650496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f88380fe9b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8834009990 tx=0x7f8834009ca0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 -- 172.21.15.6:0/368650496 shutdown_connections 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 --2- 172.21.15.6:0/368650496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f88380fe9b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 --2- 172.21.15.6:0/368650496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 0x7f883810d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 --2- 172.21.15.6:0/368650496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 0x7f8838104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 -- 172.21.15.6:0/368650496 >> 172.21.15.6:0/368650496 conn(0x7f88380753b0 msgr2=0x7f88380757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 -- 172.21.15.6:0/368650496 shutdown_connections 2023-11-17T00:16:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.112+0000 7f883ffff700 1 -- 172.21.15.6:0/368650496 wait complete. 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.114+0000 7f883ffff700 1 Processor -- start 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.114+0000 7f883ffff700 1 -- start start 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 0x7f883819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 0x7f8838198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838199260 con 0x7f88380fe5e0 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381993d0 con 0x7f88380fef80 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838199590 con 0x7f8838105680 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59574/0 (socket says 172.21.15.6:59574) 2023-11-17T00:16:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 -- 172.21.15.6:0/3164014317 learned_addr learned my addr 172.21.15.6:0/3164014317 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883d59a700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 0x7f883819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883e59c700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 0x7f8838198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 -- 172.21.15.6:0/3164014317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 msgr2=0x7f8838198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 0x7f8838198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 -- 172.21.15.6:0/3164014317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 msgr2=0x7f883819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 0x7f883819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.115+0000 7f883dd9b700 1 -- 172.21.15.6:0/3164014317 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340095f0 con 0x7f88380fe5e0 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f883dd9b700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8834005910 tx=0x7f8834005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f882effd700 1 -- 172.21.15.6:0/3164014317 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883401d070 con 0x7f88380fe5e0 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f882effd700 1 -- 172.21.15.6:0/3164014317 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8834022470 con 0x7f88380fe5e0 2023-11-17T00:16:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f882effd700 1 -- 172.21.15.6:0/3164014317 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834017650 con 0x7f88380fe5e0 2023-11-17T00:16:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8838100620 con 0x7f88380fe5e0 2023-11-17T00:16:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.116+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838100490 con 0x7f88380fe5e0 2023-11-17T00:16:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.120+0000 7f882effd700 1 -- 172.21.15.6:0/3164014317 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f88340225e0 con 0x7f88380fe5e0 2023-11-17T00:16:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 Processor -- start 2023-11-17T00:16:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 -- start start 2023-11-17T00:16:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f882effd700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 0x7f88240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5630749506b0 0x563074afa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f882effd700 1 -- 172.21.15.6:0/3164014317 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88340ce420 con 0x7f88380fe5e0 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074af4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630749fa000 con 0x563074afa720 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074a4c840 con 0x5630749506b0 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.121+0000 7f8853954b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074a45cd0 con 0x563074afab90 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883e59c700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 0x7f88240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59578/0 (socket says 172.21.15.6:59578) 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 -- 172.21.15.6:0/1662059180 learned_addr learned my addr 172.21.15.6:0/1662059180 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883e59c700 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074af4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 -- 172.21.15.6:0/1662059180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 msgr2=0x563074af4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074af4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 -- 172.21.15.6:0/1662059180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5630749506b0 msgr2=0x563074afa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5630749506b0 0x563074afa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 -- 172.21.15.6:0/1662059180 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630748c75d0 con 0x563074afa720 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883e59c700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 0x7f88240a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8830009ec0 tx=0x7f8830013250 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.122+0000 7f883d59a700 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8828009cf0 tx=0x7f8828011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eb28214183c9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.123+0000 7f8823fff700 1 -- 172.21.15.6:0/1662059180 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828011e30 con 0x563074afa720 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.123+0000 7f8823fff700 1 -- 172.21.15.6:0/1662059180 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8828004510 con 0x563074afa720 2023-11-17T00:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.123+0000 7f8823fff700 1 -- 172.21.15.6:0/1662059180 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828005280 con 0x563074afa720 2023-11-17T00:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.123+0000 7f8853954b80 1 -- 172.21.15.6:0/1662059180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 msgr2=0x563074af4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.123+0000 7f8853954b80 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8828009cf0 tx=0x7f8828011440 comp rx=0 tx=0).stop 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 -- 172.21.15.6:0/1662059180 shutdown_connections 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074af4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5630749506b0 0x563074afa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 --2- 172.21.15.6:0/1662059180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af4560 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 -- 172.21.15.6:0/1662059180 >> 172.21.15.6:0/1662059180 conn(0x563074ae9f50 msgr2=0x563074aef270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 -- 172.21.15.6:0/1662059180 shutdown_connections 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 -- 172.21.15.6:0/1662059180 wait complete. 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.124+0000 7f8853954b80 1 -- start start 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.125+0000 7f8853954b80 1 Processor -- start 2023-11-17T00:16:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afa720 0x563074afdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074afe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630749e7880 con 0x5630749506b0 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630749e7520 con 0x563074afa720 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630748c98c0 con 0x563074afab90 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f8853954b80 10 client.0 _ll_register_callbacks cb 0x563074a05720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59588/0 (socket says 172.21.15.6:59588) 2023-11-17T00:16:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 -- 172.21.15.6:0/4127196152 learned_addr learned my addr 172.21.15.6:0/4127196152 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883e59c700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074afe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883d59a700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afa720 0x563074afdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 -- 172.21.15.6:0/4127196152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 msgr2=0x563074afe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074afe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 -- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afa720 msgr2=0x563074afdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afa720 0x563074afdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.126+0000 7f883dd9b700 1 -- 172.21.15.6:0/4127196152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8828009920 con 0x5630749506b0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f883dd9b700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88340a3010 tx=0x7f88340974c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a32e4a41dcdf7a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f88217fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883409af50 con 0x5630749506b0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f883409b180 con 0x5630749506b0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563074ae7710 con 0x5630749506b0 2023-11-17T00:16:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883409acc0 con 0x5630749506b0 2023-11-17T00:16:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f8853954b80 10 client.16581 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563074a0c130 con 0x5630749506b0 2023-11-17T00:16:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.127+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88340b7080 con 0x5630749506b0 2023-11-17T00:16:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f883409a700 con 0x5630749506b0 2023-11-17T00:16:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 20 client.16581 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b0db40 0x563074b0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 4 client.16581 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563074a08040 con 0x563074b0db40 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.128+0000 7f883d59a700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b0db40 0x563074b0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.129+0000 7f883d59a700 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b0db40 0x563074b0ff20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3cac9b21ddd404 server_cookie=2476662fa08480ee in_seq=0 out_seq=0 2023-11-17T00:16:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.129+0000 7f88217fa700 10 client.16581 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f88217fa700 1 -- 172.21.15.6:0/4127196152 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563074a08040 con 0x563074b0db40 2023-11-17T00:16:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f88217fa700 10 client.16581 handle_command_reply: tid=0 2023-11-17T00:16:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 client.16581 shutdown 2023-11-17T00:16:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 msgr2=0x563074b00aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88340a3010 tx=0x7f88340974c0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 shutdown_connections 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b0db40 0x563074b0ff20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5630749506b0 0x563074b00aa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563074afab90 0x563074afe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 --2- 172.21.15.6:0/4127196152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afa720 0x563074afdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 >> 172.21.15.6:0/4127196152 conn(0x563074ae9f50 msgr2=0x563074aee7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f88217fa700 0 client.16581 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 shutdown_connections 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 1 -- 172.21.15.6:0/4127196152 wait complete. 2023-11-17T00:16:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.130+0000 7f8853954b80 20 client.16581 trim_cache size 0 max 16384 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 Processor -- start 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 -- start start 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074ae9b70 con 0x563074afa720 2023-11-17T00:16:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074af9df0 con 0x563074afab90 2023-11-17T00:16:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.146+0000 7f8853954b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074afc280 con 0x5630749506b0 2023-11-17T00:16:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883d59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883d59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59594/0 (socket says 172.21.15.6:59594) 2023-11-17T00:16:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883d59a700 1 -- 172.21.15.6:0/2467928477 learned_addr learned my addr 172.21.15.6:0/2467928477 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883e59c700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 -- 172.21.15.6:0/2467928477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 msgr2=0x563074af4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 -- 172.21.15.6:0/2467928477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 msgr2=0x563074b00640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 -- 172.21.15.6:0/2467928477 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563074aeb6f0 con 0x5630749506b0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883d59a700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.147+0000 7f883dd9b700 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b05060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f883400b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe63f43aaef8f09 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f882cff9700 1 -- 172.21.15.6:0/2467928477 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883409acc0 con 0x5630749506b0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f882cff9700 1 -- 172.21.15.6:0/2467928477 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563074ae7710 con 0x5630749506b0 2023-11-17T00:16:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f882cff9700 1 -- 172.21.15.6:0/2467928477 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340a3540 con 0x5630749506b0 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 -- 172.21.15.6:0/2467928477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 msgr2=0x563074b05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b05060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f883400b490 comp rx=0 tx=0).stop 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 -- 172.21.15.6:0/2467928477 shutdown_connections 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b05060 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 --2- 172.21.15.6:0/2467928477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074b00640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 -- 172.21.15.6:0/2467928477 >> 172.21.15.6:0/2467928477 conn(0x563074ae9f50 msgr2=0x563074af7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 -- 172.21.15.6:0/2467928477 shutdown_connections 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.148+0000 7f8853954b80 1 -- 172.21.15.6:0/2467928477 wait complete. 2023-11-17T00:16:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 -- start start 2023-11-17T00:16:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 Processor -- start 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b04f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074af7200 con 0x563074afa720 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074af7340 con 0x563074afab90 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.149+0000 7f8853954b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563074af7500 con 0x5630749506b0 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f8853954b80 10 client.0 _ll_register_callbacks cb 0x7f88340057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b04f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883dd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b04f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41234/0 (socket says 172.21.15.6:41234) 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883dd9b700 1 -- 172.21.15.6:0/1088069858 learned_addr learned my addr 172.21.15.6:0/1088069858 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:41806/0 (socket says 172.21.15.6:41806) 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 -- 172.21.15.6:0/1088069858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 msgr2=0x563074b04f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b04f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 -- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 msgr2=0x563074af6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 -- 172.21.15.6:0/1088069858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882801d040 con 0x563074afa720 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f883d59a700 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af6780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563074ae9e70 tx=0x7f8828004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15faf867c7a95e43 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.150+0000 7f88217fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828004050 con 0x563074afa720 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8828004510 con 0x563074afa720 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8828029070 con 0x563074afa720 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88280041b0 con 0x563074afa720 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f8853954b80 10 client.16587 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563074a0c130 con 0x563074afa720 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8828004510 con 0x563074afa720 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.151+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8828033060 con 0x563074afa720 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 20 client.16587 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b61ee0 0x563074b642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 4 client.16587 mds_command: new command op to 24578 tid=0[{"prefix": "flush journal"}] 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563074a14600 con 0x563074b61ee0 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f883dd9b700 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b61ee0 0x563074b642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f883dd9b700 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b61ee0 0x563074b642f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e7668fa6622182 server_cookie=8d3dd43f777e3547 in_seq=0 out_seq=0 2023-11-17T00:16:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.152+0000 7f88217fa700 10 client.16587 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f88217fa700 1 -- 172.21.15.6:0/1088069858 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563074a14600 con 0x563074b61ee0 2023-11-17T00:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f88217fa700 10 client.16587 handle_command_reply: tid=0 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 client.16587 shutdown 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 msgr2=0x563074af6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af6780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563074ae9e70 tx=0x7f8828004970 comp rx=0 tx=0).stop 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 shutdown_connections 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563074afab90 0x563074af6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5630749506b0 0x563074b04f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x563074b61ee0 0x563074b642f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 --2- 172.21.15.6:0/1088069858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x563074afa720 0x563074af6780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f88217fa700 0 client.16587 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 >> 172.21.15.6:0/1088069858 conn(0x563074ae9f50 msgr2=0x563074af8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 shutdown_connections 2023-11-17T00:16:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.171+0000 7f8853954b80 1 -- 172.21.15.6:0/1088069858 wait complete. 2023-11-17T00:16:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f8853954b80 20 client.16587 trim_cache size 0 max 16384 2023-11-17T00:16:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:{ 2023-11-17T00:16:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "message": "", 2023-11-17T00:16:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout: "return_code": 0 2023-11-17T00:16:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stdout:} 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 msgr2=0x7f88240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 0x7f88240a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8830009ec0 tx=0x7f8830013250 comp rx=0 tx=0).stop 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 msgr2=0x7f883819d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8834005910 tx=0x7f8834005720 comp rx=0 tx=0).stop 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 shutdown_connections 2023-11-17T00:16:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f88240a12c0 0x7f88240a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88380fe5e0 0x7f883819d230 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8838105680 0x7f8838198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 --2- 172.21.15.6:0/3164014317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88380fef80 0x7f883819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.172+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 >> 172.21.15.6:0/3164014317 conn(0x7f88380753b0 msgr2=0x7f88380fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.173+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 shutdown_connections 2023-11-17T00:16:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi006.front.sepia.ceph.com.smithi006.stderr:2023-11-17T00:16:39.173+0000 7f883ffff700 1 -- 172.21.15.6:0/3164014317 wait complete. 2023-11-17T00:16:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:38 smithi006 ceph-mon[97840]: pgmap v239: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 189 op/s 2023-11-17T00:16:39.253 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-17T00:16:39.363 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c4ef3c3277f54ce6ad59166747e24568 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_1c9565146c8647d4e00eb6c7ae626dc192680e7b/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-17T00:16:39.366 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-11-17T00:16:39.376 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:16:39.380 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:38 smithi186 ceph-mon[131487]: pgmap v239: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 189 op/s 2023-11-17T00:16:39.381 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:38 smithi087 ceph-mon[131182]: pgmap v239: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 189 op/s 2023-11-17T00:16:39.388 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-11-17T00:16:39.398 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-11-17T00:16:39.398 INFO:teuthology.task.exec:Executing custom commands... 2023-11-17T00:16:39.398 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:16:39.399 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-11-17T00:16:39.511 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-11-17T00:16:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:40 smithi006 ceph-mon[97840]: pgmap v240: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 KiB/s wr, 168 op/s 2023-11-17T00:16:41.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:40 smithi087 ceph-mon[131182]: pgmap v240: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 KiB/s wr, 168 op/s 2023-11-17T00:16:41.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:40 smithi186 ceph-mon[131487]: pgmap v240: 129 pgs: 129 active+clean; 4.7 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 KiB/s wr, 168 op/s 2023-11-17T00:16:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:42 smithi006 ceph-mon[97840]: pgmap v241: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 48 KiB/s wr, 174 op/s 2023-11-17T00:16:43.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:42 smithi087 ceph-mon[131182]: pgmap v241: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 48 KiB/s wr, 174 op/s 2023-11-17T00:16:43.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:42 smithi186 ceph-mon[131487]: pgmap v241: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 48 KiB/s wr, 174 op/s 2023-11-17T00:16:45.159 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:16:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:44 smithi006 ceph-mon[97840]: pgmap v242: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-11-17T00:16:45.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:44 smithi087 ceph-mon[131182]: pgmap v242: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-11-17T00:16:45.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:44 smithi186 ceph-mon[131487]: pgmap v242: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-11-17T00:16:45.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 -- 172.21.15.6:0/455837835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8100c50 msgr2=0x7feda8101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 --2- 172.21.15.6:0/455837835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8100c50 0x7feda8101020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed98009990 tx=0x7fed98009ca0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 -- 172.21.15.6:0/455837835 shutdown_connections 2023-11-17T00:16:45.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 --2- 172.21.15.6:0/455837835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda8105560 0x7feda810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 --2- 172.21.15.6:0/455837835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8100c50 0x7feda8101020 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 --2- 172.21.15.6:0/455837835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda81015f0 0x7feda8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 -- 172.21.15.6:0/455837835 >> 172.21.15.6:0/455837835 conn(0x7feda8077500 msgr2=0x7feda8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 -- 172.21.15.6:0/455837835 shutdown_connections 2023-11-17T00:16:45.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.378+0000 7fedad9ff700 1 -- 172.21.15.6:0/455837835 wait complete. 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 Processor -- start 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 -- start start 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 0x7feda8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8105560 0x7feda819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda819d550 con 0x7feda81015f0 2023-11-17T00:16:45.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda819d6c0 con 0x7feda8100c50 2023-11-17T00:16:45.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.379+0000 7fedad9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda819d880 con 0x7feda8105560 2023-11-17T00:16:45.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 0x7feda8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:41828/0 (socket says 172.21.15.6:41828) 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda6ffd700 1 -- 172.21.15.6:0/3447542091 learned_addr learned my addr 172.21.15.6:0/3447542091 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda77fe700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8105560 0x7feda819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 -- 172.21.15.6:0/3447542091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8105560 msgr2=0x7feda819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8105560 0x7feda819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 -- 172.21.15.6:0/3447542091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 msgr2=0x7feda8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 -- 172.21.15.6:0/3447542091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c009520 con 0x7feda81015f0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda6ffd700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7feda67fc700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 0x7feda8198ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feda81019c0 tx=0x7fed9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7fedac9fd700 1 -- 172.21.15.6:0/3447542091 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c004220 con 0x7feda81015f0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7fedac9fd700 1 -- 172.21.15.6:0/3447542091 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c004380 con 0x7feda81015f0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.380+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda0009c40 con 0x7feda81015f0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.381+0000 7fedac9fd700 1 -- 172.21.15.6:0/3447542091 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c003c30 con 0x7feda81015f0 2023-11-17T00:16:45.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.381+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed980095f0 con 0x7feda81015f0 2023-11-17T00:16:45.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.384+0000 7fedac9fd700 1 -- 172.21.15.6:0/3447542091 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fed9c00c760 con 0x7feda81015f0 2023-11-17T00:16:45.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.385+0000 7fedac9fd700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 0x7fed900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.385+0000 7fedc1363b80 1 Processor -- start 2023-11-17T00:16:45.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.385+0000 7fedac9fd700 1 -- 172.21.15.6:0/3447542091 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed9c014070 con 0x7feda81015f0 2023-11-17T00:16:45.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.385+0000 7feda6ffd700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 0x7fed900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.385+0000 7fedc1363b80 1 -- start start 2023-11-17T00:16:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d55ea060 con 0x5600d56e4b60 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d563c8a0 con 0x5600d56e98b0 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7fedc1363b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d5635d30 con 0x5600d56e5470 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59638/0 (socket says 172.21.15.6:59638) 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda6ffd700 1 -- 172.21.15.6:0/783547081 learned_addr learned my addr 172.21.15.6:0/783547081 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda67fc700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda6ffd700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 0x7fed900a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed98006010 tx=0x7fed98005960 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.386+0000 7feda77fe700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 -- 172.21.15.6:0/783547081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 msgr2=0x5600d56e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56e9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 -- 172.21.15.6:0/783547081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 msgr2=0x5600d56e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 -- 172.21.15.6:0/783547081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600d54b7630 con 0x5600d56e5470 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda77fe700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda6ffd700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:45.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7feda67fc700 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56e9370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ca8f0 tx=0x7fed9c099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ad4b13658504da server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7fed8dffb700 1 -- 172.21.15.6:0/783547081 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c09a270 con 0x5600d56e5470 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7fed8dffb700 1 -- 172.21.15.6:0/783547081 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c09e2a0 con 0x5600d56e5470 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.387+0000 7fed8dffb700 1 -- 172.21.15.6:0/783547081 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00cc10 con 0x5600d56e5470 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 -- 172.21.15.6:0/783547081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 msgr2=0x5600d56e9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56e9370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ca8f0 tx=0x7fed9c099e30 comp rx=0 tx=0).stop 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 -- 172.21.15.6:0/783547081 shutdown_connections 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 --2- 172.21.15.6:0/783547081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56e9370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 -- 172.21.15.6:0/783547081 >> 172.21.15.6:0/783547081 conn(0x5600d56da360 msgr2=0x5600d56dfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 -- 172.21.15.6:0/783547081 shutdown_connections 2023-11-17T00:16:45.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.388+0000 7fedc1363b80 1 -- 172.21.15.6:0/783547081 wait complete. 2023-11-17T00:16:45.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 -- start start 2023-11-17T00:16:45.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 Processor -- start 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d55d78e0 con 0x5600d56e4b60 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d55d7580 con 0x5600d56e98b0 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d54b9920 con 0x5600d56e5470 2023-11-17T00:16:45.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.389+0000 7fedc1363b80 10 client.0 _ll_register_callbacks cb 0x5600d55f5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59654/0 (socket says 172.21.15.6:59654) 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 -- 172.21.15.6:0/1800401922 learned_addr learned my addr 172.21.15.6:0/1800401922 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda77fe700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda67fc700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 -- 172.21.15.6:0/1800401922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 msgr2=0x5600d56ee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56ee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 -- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 msgr2=0x5600d56eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 -- 172.21.15.6:0/1800401922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed980095f0 con 0x5600d56e4b60 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7feda6ffd700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b6f0 tx=0x7fed9801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c1b80ba3695c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.390+0000 7fed86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98028030 con 0x5600d56e4b60 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed98019aa0 con 0x5600d56e4b60 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600d56d7b20 con 0x5600d56e4b60 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed98019c60 con 0x5600d56e4b60 2023-11-17T00:16:45.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fedc1363b80 10 client.16599 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5600d55fc190 con 0x5600d56e4b60 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600d56d7b20 con 0x5600d56e4b60 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed98024410 con 0x5600d56e4b60 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fedc1363b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.391+0000 7fedc1363b80 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.392+0000 7fedc1363b80 20 client.16599 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:45.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.392+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d56fe880 0x5600d5700c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.392+0000 7fedc1363b80 4 client.16599 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.392+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600d55f80a0 con 0x5600d56fe880 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.392+0000 7feda67fc700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d56fe880 0x5600d5700c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.393+0000 7fed86ffd700 10 client.16599 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.393+0000 7feda67fc700 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d56fe880 0x5600d5700c90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35936c6a6d18908a server_cookie=caa3fd177ded8fa4 in_seq=0 out_seq=0 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.393+0000 7fed86ffd700 1 -- 172.21.15.6:0/1800401922 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5600d55f80a0 con 0x5600d56fe880 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.393+0000 7fed86ffd700 10 client.16599 handle_command_reply: tid=0 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.393+0000 7fedc1363b80 1 client.16599 shutdown 2023-11-17T00:16:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 msgr2=0x5600d56f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fed9800b6f0 tx=0x7fed9801a050 comp rx=0 tx=0).stop 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 shutdown_connections 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d56fe880 0x5600d5700c90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56f1060 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e98b0 0x5600d56eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 --2- 172.21.15.6:0/1800401922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e5470 0x5600d56ee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fed86ffd700 0 client.16599 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:45.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 >> 172.21.15.6:0/1800401922 conn(0x5600d56da360 msgr2=0x5600d56ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 shutdown_connections 2023-11-17T00:16:45.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 1 -- 172.21.15.6:0/1800401922 wait complete. 2023-11-17T00:16:45.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.394+0000 7fedc1363b80 20 client.16599 trim_cache size 0 max 16384 2023-11-17T00:16:45.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 Processor -- start 2023-11-17T00:16:45.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 -- start start 2023-11-17T00:16:45.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 0x5600d56f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56e08b0 con 0x5600d56e4b60 2023-11-17T00:16:45.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56e5bc0 con 0x5600d56e5470 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7fedc1363b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56ef8f0 con 0x5600d56e98b0 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59656/0 (socket says 172.21.15.6:59656) 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.412+0000 7feda6ffd700 1 -- 172.21.15.6:0/185867104 learned_addr learned my addr 172.21.15.6:0/185867104 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda67fc700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 0x5600d56f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda77fe700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 -- 172.21.15.6:0/185867104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 msgr2=0x5600d56edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 -- 172.21.15.6:0/185867104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 msgr2=0x5600d56f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 0x5600d56f0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 -- 172.21.15.6:0/185867104 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600d56e9f50 con 0x5600d56e4b60 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda67fc700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 0x5600d56f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7feda6ffd700 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980248f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc5de72120950f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7fed8e7fc700 1 -- 172.21.15.6:0/185867104 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98028040 con 0x5600d56e4b60 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7fed8e7fc700 1 -- 172.21.15.6:0/185867104 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9801e040 con 0x5600d56e4b60 2023-11-17T00:16:45.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.413+0000 7fed8e7fc700 1 -- 172.21.15.6:0/185867104 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98019bf0 con 0x5600d56e4b60 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- 172.21.15.6:0/185867104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 msgr2=0x5600d56de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980248f0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- 172.21.15.6:0/185867104 shutdown_connections 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e4b60 0x5600d56de550 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 --2- 172.21.15.6:0/185867104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e5470 0x5600d56f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- 172.21.15.6:0/185867104 >> 172.21.15.6:0/185867104 conn(0x5600d56da360 msgr2=0x5600d56fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- 172.21.15.6:0/185867104 shutdown_connections 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- 172.21.15.6:0/185867104 wait complete. 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.414+0000 7fedc1363b80 1 -- start start 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 Processor -- start 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 0x5600d574b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56f3ed0 con 0x5600d56e5470 2023-11-17T00:16:45.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56f4040 con 0x5600d56e4b60 2023-11-17T00:16:45.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600d56f4200 con 0x5600d56e98b0 2023-11-17T00:16:45.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7fedc1363b80 10 client.0 _ll_register_callbacks cb 0x7fed9800f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:45.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.415+0000 7feda67fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59666/0 (socket says 172.21.15.6:59666) 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda6ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 0x5600d574b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 -- 172.21.15.6:0/532087240 learned_addr learned my addr 172.21.15.6:0/532087240 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 -- 172.21.15.6:0/532087240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 msgr2=0x5600d56f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 -- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 msgr2=0x5600d574b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:45.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 0x5600d574b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 -- 172.21.15.6:0/532087240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600d54b7630 con 0x5600d56e5470 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda6ffd700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 0x5600d574b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7feda67fc700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed9c096770 tx=0x7fed9c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1131188a17c0092d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7fed86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0a2740 con 0x5600d56e5470 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed9c00cc20 con 0x5600d56e5470 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.416+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed9c09e2a0 con 0x5600d56e5470 2023-11-17T00:16:45.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c099db0 con 0x5600d56e5470 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 10 client.16605 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5600d55fc190 con 0x5600d56e5470 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c09e2a0 con 0x5600d56e5470 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed9c065030 con 0x5600d56e5470 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 20 client.16605 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:45.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d57533a0 0x5600d57557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 4 client.16605 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:16:45.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600d5604660 con 0x5600d57533a0 2023-11-17T00:16:45.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.417+0000 7feda6ffd700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d57533a0 0x5600d57557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7feda6ffd700 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d57533a0 0x5600d57557b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1df612eb96a106 server_cookie=ab8df911bd80b0b8 in_seq=0 out_seq=0 2023-11-17T00:16:45.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fed86ffd700 10 client.16605 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:45.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fed86ffd700 1 -- 172.21.15.6:0/532087240 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5600d5604660 con 0x5600d57533a0 2023-11-17T00:16:45.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fed86ffd700 10 client.16605 handle_command_reply: tid=0 2023-11-17T00:16:45.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 client.16605 shutdown 2023-11-17T00:16:45.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 msgr2=0x5600d574beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed9c096770 tx=0x7fed9c003680 comp rx=0 tx=0).stop 2023-11-17T00:16:45.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 shutdown_connections 2023-11-17T00:16:45.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d56e98b0 0x5600d56f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5600d56e4b60 0x5600d574b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5600d57533a0 0x5600d57557b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 --2- 172.21.15.6:0/532087240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5600d56e5470 0x5600d574beb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 >> 172.21.15.6:0/532087240 conn(0x5600d56da360 msgr2=0x5600d56de550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fed86ffd700 0 client.16605 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 shutdown_connections 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.418+0000 7fedc1363b80 1 -- 172.21.15.6:0/532087240 wait complete. 2023-11-17T00:16:45.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedc1363b80 20 client.16605 trim_cache size 0 max 16384 2023-11-17T00:16:45.534 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:45.534 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:16:45.534 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:16:45.534 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:45.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 msgr2=0x7fed900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 0x7fed900a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed98006010 tx=0x7fed98005960 comp rx=0 tx=0).stop 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 msgr2=0x7feda8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 0x7feda8198ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feda81019c0 tx=0x7fed9c00eca0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 shutdown_connections 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fed900a12c0 0x7fed900a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feda8105560 0x7feda819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda8100c50 0x7feda8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 --2- 172.21.15.6:0/3447542091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feda81015f0 0x7feda8198ab0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 >> 172.21.15.6:0/3447542091 conn(0x7feda8077500 msgr2=0x7feda80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 shutdown_connections 2023-11-17T00:16:45.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.419+0000 7fedad9ff700 1 -- 172.21.15.6:0/3447542091 wait complete. 2023-11-17T00:16:45.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:16:45.537 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:16:45.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 -- 172.21.15.6:0/3780341212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc105380 msgr2=0x7f29bc10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 --2- 172.21.15.6:0/3780341212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc105380 0x7f29bc10cf50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009fc0 tx=0x7f29b400b330 comp rx=0 tx=0).stop 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 -- 172.21.15.6:0/3780341212 shutdown_connections 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 --2- 172.21.15.6:0/3780341212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc105380 0x7f29bc10cf50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 --2- 172.21.15.6:0/3780341212 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc100340 0x7f29bc100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 --2- 172.21.15.6:0/3780341212 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100ce0 0x7f29bc104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 -- 172.21.15.6:0/3780341212 >> 172.21.15.6:0/3780341212 conn(0x7f29bc0fbbd0 msgr2=0x7f29bc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:45.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 -- 172.21.15.6:0/3780341212 shutdown_connections 2023-11-17T00:16:45.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.743+0000 7f29c319d700 1 -- 172.21.15.6:0/3780341212 wait complete. 2023-11-17T00:16:45.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.744+0000 7f29c319d700 1 Processor -- start 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.744+0000 7f29c319d700 1 -- start start 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 0x7f29bc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc105380 0x7f29bc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19d480 con 0x7f29bc100340 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19d5f0 con 0x7f29bc105380 2023-11-17T00:16:45.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c319d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19d7b0 con 0x7f29bc100ce0 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59678/0 (socket says 172.21.15.6:59678) 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 -- 172.21.15.6:0/3125833899 learned_addr learned my addr 172.21.15.6:0/3125833899 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c199a700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 0x7f29bc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c299c700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc105380 0x7f29bc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 -- 172.21.15.6:0/3125833899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 msgr2=0x7f29bc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 0x7f29bc198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 -- 172.21.15.6:0/3125833899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc105380 msgr2=0x7f29bc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc105380 0x7f29bc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c219b700 1 -- 172.21.15.6:0/3125833899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b4009c40 con 0x7f29bc100340 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.745+0000 7f29c199a700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 0x7f29bc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29c219b700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29b800e9a0 tx=0x7f29b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8009790 con 0x7f29bc100340 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc1101a0 con 0x7f29bc100340 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b800cbd0 con 0x7f29bc100340 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc110010 con 0x7f29bc100340 2023-11-17T00:16:45.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.746+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8018620 con 0x7f29bc100340 2023-11-17T00:16:45.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.747+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc110330 con 0x7f29bc100340 2023-11-17T00:16:45.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.749+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f29b800cd40 con 0x7f29bc100340 2023-11-17T00:16:45.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.751+0000 7f29b37fe700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 0x7f29a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:45.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.751+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f29b8014070 con 0x7f29bc100340 2023-11-17T00:16:45.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.751+0000 7f29c199a700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 0x7f29a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:45.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.752+0000 7f29c199a700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 0x7f29a80a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f29bc199b50 tx=0x7f29ac009310 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:45.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:45.755+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f29b8051730 con 0x7f29bc100340 2023-11-17T00:16:46.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.008+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29bc067970 con 0x7f29bc100340 2023-11-17T00:16:46.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.009+0000 7f29b37fe700 1 -- 172.21.15.6:0/3125833899 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7f29b8099ed0 con 0x7f29bc100340 2023-11-17T00:16:46.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:16:46.010 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:16:46.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 msgr2=0x7f29a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 0x7f29a80a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f29bc199b50 tx=0x7f29ac009310 comp rx=0 tx=0).stop 2023-11-17T00:16:46.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 msgr2=0x7f29bc198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29b800e9a0 tx=0x7f29b800ed60 comp rx=0 tx=0).stop 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 shutdown_connections 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f29a80a12c0 0x7f29a80a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f29bc100340 0x7f29bc198540 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29bc105380 0x7f29bc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 --2- 172.21.15.6:0/3125833899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29bc100ce0 0x7f29bc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 >> 172.21.15.6:0/3125833899 conn(0x7f29bc0fbbd0 msgr2=0x7f29bc0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 shutdown_connections 2023-11-17T00:16:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.011+0000 7f29c319d700 1 -- 172.21.15.6:0/3125833899 wait complete. 2023-11-17T00:16:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:16:46.032 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:16:46.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 -- 172.21.15.6:0/248173644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c701015f0 msgr2=0x7f9c70104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 --2- 172.21.15.6:0/248173644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c701015f0 0x7f9c70104e80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9c640098f0 tx=0x7f9c64009c00 comp rx=0 tx=0).stop 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 -- 172.21.15.6:0/248173644 shutdown_connections 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 --2- 172.21.15.6:0/248173644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c70105560 0x7f9c7010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 --2- 172.21.15.6:0/248173644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 --2- 172.21.15.6:0/248173644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c701015f0 0x7f9c70104e80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 -- 172.21.15.6:0/248173644 >> 172.21.15.6:0/248173644 conn(0x7f9c70077500 msgr2=0x7f9c70077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 -- 172.21.15.6:0/248173644 shutdown_connections 2023-11-17T00:16:46.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.239+0000 7f9c765ff700 1 -- 172.21.15.6:0/248173644 wait complete. 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 Processor -- start 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 -- start start 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c701015f0 0x7f9c70198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 0x7f9c7019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019d5e0 con 0x7f9c701015f0 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019d750 con 0x7f9c70100c50 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c765ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019d910 con 0x7f9c70105560 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.241+0000 7f9c74b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 0x7f9c7019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:41908/0 (socket says 172.21.15.6:41908) 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3176837318 learned_addr learned my addr 172.21.15.6:0/3176837318 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c701015f0 0x7f9c70198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3176837318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 msgr2=0x7f9c7019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 0x7f9c7019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c701015f0 msgr2=0x7f9c70198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c701015f0 0x7f9c70198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3176837318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c64009520 con 0x7f9c70100c50 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c74b9c700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 0x7f9c7019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000e9a0 tx=0x7f9c6000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6d7fa700 1 -- 172.21.15.6:0/3176837318 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60009790 con 0x7f9c70100c50 2023-11-17T00:16:46.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c70110320 con 0x7f9c70100c50 2023-11-17T00:16:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6d7fa700 1 -- 172.21.15.6:0/3176837318 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c60010430 con 0x7f9c70100c50 2023-11-17T00:16:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.242+0000 7f9c6d7fa700 1 -- 172.21.15.6:0/3176837318 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60018680 con 0x7f9c70100c50 2023-11-17T00:16:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.243+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c70110190 con 0x7f9c70100c50 2023-11-17T00:16:46.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.246+0000 7f9c6d7fa700 1 -- 172.21.15.6:0/3176837318 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9c600105a0 con 0x7f9c70100c50 2023-11-17T00:16:46.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.247+0000 7f9c89e7cb80 1 Processor -- start 2023-11-17T00:16:46.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.247+0000 7f9c89e7cb80 1 -- start start 2023-11-17T00:16:46.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.247+0000 7f9c6d7fa700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 0x7f9c580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c6d7fa700 1 -- 172.21.15.6:0/3176837318 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c60014070 con 0x7f9c70100c50 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c74b9c700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 0x7f9c580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f93c060 con 0x55968fa3f370 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f98e8a0 con 0x55968fa3fc80 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f987d30 con 0x55968fa440c0 2023-11-17T00:16:46.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59708/0 (socket says 172.21.15.6:59708) 2023-11-17T00:16:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3974033089 learned_addr learned my addr 172.21.15.6:0/3974033089 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.248+0000 7f9c74b9c700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c74b9c700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 0x7f9c580a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003e40 tx=0x7f9c680099e0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3974033089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 msgr2=0x55968fa44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3974033089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 msgr2=0x55968fa43b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa43b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 -- 172.21.15.6:0/3974033089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55968f809630 con 0x55968fa3f370 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa43b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:46.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c60096a20 tx=0x7f9c60099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e28cb4417cf2fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c567fc700 1 -- 172.21.15.6:0/3974033089 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600a2400 con 0x55968fa3f370 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c567fc700 1 -- 172.21.15.6:0/3974033089 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c60010880 con 0x55968fa3f370 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.249+0000 7f9c567fc700 1 -- 172.21.15.6:0/3974033089 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600996e0 con 0x55968fa3f370 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/3974033089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 msgr2=0x55968fa3f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c60096a20 tx=0x7f9c60099c20 comp rx=0 tx=0).stop 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/3974033089 shutdown_connections 2023-11-17T00:16:46.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa3f740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/3974033089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa43b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/3974033089 >> 172.21.15.6:0/3974033089 conn(0x55968fa2c360 msgr2=0x55968fa31b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/3974033089 shutdown_connections 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.250+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/3974033089 wait complete. 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 -- start start 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 Processor -- start 2023-11-17T00:16:46.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 0x55968fa48b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa440c0 0x55968fa49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f9298e0 con 0x55968fa3f370 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f929580 con 0x55968fa440c0 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968f80b920 con 0x55968fa3fc80 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c89e7cb80 10 client.0 _ll_register_callbacks cb 0x55968f947780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.251+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59720/0 (socket says 172.21.15.6:59720) 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 -- 172.21.15.6:0/774709434 learned_addr learned my addr 172.21.15.6:0/774709434 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 0x55968fa48b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c74b9c700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa440c0 0x55968fa49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 -- 172.21.15.6:0/774709434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 msgr2=0x55968fa48b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 0x55968fa48b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 -- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa440c0 msgr2=0x55968fa49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa440c0 0x55968fa49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 -- 172.21.15.6:0/774709434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c70110190 con 0x55968fa3f370 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 0x55968fa48b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c60099b30 tx=0x7f9c6009a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233019d4491f9aac server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600a2400 con 0x55968fa3f370 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c600a2180 con 0x55968fa3f370 2023-11-17T00:16:46.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.252+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55968fa29b20 con 0x55968fa3f370 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c60094bc0 con 0x55968fa3f370 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 10 client.16623 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55968f94e190 con 0x55968fa3f370 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55968fa29b20 con 0x55968fa3f370 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c60098fa0 con 0x55968fa3f370 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 10 client.16623 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:46.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 20 client.16623 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:46.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968fa58ca0 0x55968fa5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 4 client.16623 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:46.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.253+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55968f94a0a0 con 0x55968fa58ca0 2023-11-17T00:16:46.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.254+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968fa58ca0 0x55968fa5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.254+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968fa58ca0 0x55968fa5b080 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704c90caa43b3578 server_cookie=6eb1bcaaf2edc438 in_seq=0 out_seq=0 2023-11-17T00:16:46.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.254+0000 7f9c4f7fe700 10 client.16623 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:46.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.255+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/774709434 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55968f94a0a0 con 0x55968fa58ca0 2023-11-17T00:16:46.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.255+0000 7f9c4f7fe700 10 client.16623 handle_command_reply: tid=0 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.255+0000 7f9c89e7cb80 1 client.16623 shutdown 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.255+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 msgr2=0x55968fa4b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.255+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c60099b30 tx=0x7f9c6009a440 comp rx=0 tx=0).stop 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 shutdown_connections 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968fa58ca0 0x55968fa5b080 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3f370 0x55968fa4b8e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa440c0 0x55968fa49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/774709434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3fc80 0x55968fa48b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c4f7fe700 0 client.16623 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:46.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 >> 172.21.15.6:0/774709434 conn(0x55968fa2c360 msgr2=0x55968fa45570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 shutdown_connections 2023-11-17T00:16:46.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/774709434 wait complete. 2023-11-17T00:16:46.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.256+0000 7f9c89e7cb80 20 client.16623 trim_cache size 0 max 16384 2023-11-17T00:16:46.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 Processor -- start 2023-11-17T00:16:46.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 -- start start 2023-11-17T00:16:46.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa31560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 0x55968fa31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968fa46c10 con 0x55968fa440c0 2023-11-17T00:16:46.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968fa4a110 con 0x55968fa3fc80 2023-11-17T00:16:46.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968fa42cf0 con 0x55968fa3f370 2023-11-17T00:16:46.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c74b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 0x55968fa31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41362/0 (socket says 172.21.15.6:41362) 2023-11-17T00:16:46.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa31560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c6ffff700 1 -- 172.21.15.6:0/2043266053 learned_addr learned my addr 172.21.15.6:0/2043266053 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:46.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.270+0000 7f9c74b9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 0x55968fa31aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59728/0 (socket says 172.21.15.6:59728) 2023-11-17T00:16:46.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 -- 172.21.15.6:0/2043266053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 msgr2=0x55968fa31560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa31560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 -- 172.21.15.6:0/2043266053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 msgr2=0x55968fa31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 0x55968fa31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 -- 172.21.15.6:0/2043266053 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55968f94dee0 con 0x55968fa3f370 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c60096a20 tx=0x7f9c60099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6245f871b65d925b server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c56ffd700 1 -- 172.21.15.6:0/2043266053 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600b8030 con 0x55968fa3f370 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c56ffd700 1 -- 172.21.15.6:0/2043266053 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c600a2400 con 0x55968fa3f370 2023-11-17T00:16:46.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.271+0000 7f9c56ffd700 1 -- 172.21.15.6:0/2043266053 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60099170 con 0x55968fa3f370 2023-11-17T00:16:46.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/2043266053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 msgr2=0x55968fa4b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c60096a20 tx=0x7f9c60099c20 comp rx=0 tx=0).stop 2023-11-17T00:16:46.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/2043266053 shutdown_connections 2023-11-17T00:16:46.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa440c0 0x55968fa31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa3f370 0x55968fa4b480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/2043266053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3fc80 0x55968fa31560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/2043266053 >> 172.21.15.6:0/2043266053 conn(0x55968fa2c360 msgr2=0x55968fa421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/2043266053 shutdown_connections 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/2043266053 wait complete. 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.272+0000 7f9c89e7cb80 1 -- start start 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 Processor -- start 2023-11-17T00:16:46.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968fa4f020 con 0x55968fa3fc80 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968faabaf0 con 0x55968fa3f370 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55968faabc60 con 0x55968fa440c0 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.273+0000 7f9c89e7cb80 10 client.0 _ll_register_callbacks cb 0x7f9c60099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59738/0 (socket says 172.21.15.6:59738) 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c74b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:41958/0 (socket says 172.21.15.6:41958) 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 -- 172.21.15.6:0/4285476060 learned_addr learned my addr 172.21.15.6:0/4285476060 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 -- 172.21.15.6:0/4285476060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 msgr2=0x55968fa4e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa4e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 -- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 msgr2=0x55968fa503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 -- 172.21.15.6:0/4285476060 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55968f94e190 con 0x55968fa3fc80 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c6f7fe700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c010 tx=0x7f9c64027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1ed428717de2fb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.274+0000 7f9c4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64027da0 con 0x55968fa3fc80 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c60010880 con 0x55968fa3fc80 2023-11-17T00:16:46.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c640271b0 con 0x55968fa3fc80 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c64027370 con 0x55968fa3fc80 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 10 client.16629 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55968f79ec60 con 0x55968fa3fc80 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c60010880 con 0x55968fa3fc80 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c64009920 con 0x55968fa3fc80 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 10 client.16629 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:46.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 20 client.16629 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968faad350 0x55968faaf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 4 client.16629 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.275+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55968f956660 con 0x55968faad350 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968faad350 0x55968faaf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c6ffff700 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968faad350 0x55968faaf760 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a372dd7d084956 server_cookie=831826264082caff in_seq=0 out_seq=0 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c4f7fe700 10 client.16629 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c4f7fe700 1 -- 172.21.15.6:0/4285476060 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55968f956660 con 0x55968faad350 2023-11-17T00:16:46.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c4f7fe700 10 client.16629 handle_command_reply: tid=0 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.276+0000 7f9c89e7cb80 1 client.16629 shutdown 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 msgr2=0x55968fa4e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c010 tx=0x7f9c64027040 comp rx=0 tx=0).stop 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 shutdown_connections 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55968fa440c0 0x55968fa4e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55968fa3f370 0x55968fa503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55968faad350 0x55968faaf760 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 --2- 172.21.15.6:0/4285476060 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55968fa3fc80 0x55968fa4e1a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 >> 172.21.15.6:0/4285476060 conn(0x55968fa2c360 msgr2=0x55968fa43100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c4f7fe700 0 client.16629 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:46.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 shutdown_connections 2023-11-17T00:16:46.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 1 -- 172.21.15.6:0/4285476060 wait complete. 2023-11-17T00:16:46.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.277+0000 7f9c89e7cb80 20 client.16629 trim_cache size 0 max 16384 2023-11-17T00:16:46.316 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:16:46.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 msgr2=0x7f9c580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 0x7f9c580a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c68003e40 tx=0x7f9c680099e0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 msgr2=0x7f9c70198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000e9a0 tx=0x7f9c6000ecb0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 shutdown_connections 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f9c580a12c0 0x7f9c580a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c70105560 0x7f9c7019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c70100c50 0x7f9c70198600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 --2- 172.21.15.6:0/3176837318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c701015f0 0x7f9c70198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:46.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 >> 172.21.15.6:0/3176837318 conn(0x7f9c70077500 msgr2=0x7f9c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:46.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 shutdown_connections 2023-11-17T00:16:46.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:46.278+0000 7f9c765ff700 1 -- 172.21.15.6:0/3176837318 wait complete. 2023-11-17T00:16:47.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:46 smithi006 ceph-mon[97840]: pgmap v243: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:47.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:46 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3125833899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:47.320 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:16:47.320 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:16:47.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:46 smithi087 ceph-mon[131182]: pgmap v243: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:47.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:46 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3125833899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:47.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:46 smithi186 ceph-mon[131487]: pgmap v243: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:47.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:46 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3125833899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:16:47.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 -- 172.21.15.6:0/3283342384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c100c50 msgr2=0x7ff16c101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 --2- 172.21.15.6:0/3283342384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c100c50 0x7ff16c101020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0099c0 tx=0x7ff15c009cd0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 -- 172.21.15.6:0/3283342384 shutdown_connections 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 --2- 172.21.15.6:0/3283342384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c105560 0x7ff16c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 --2- 172.21.15.6:0/3283342384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c100c50 0x7ff16c101020 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 --2- 172.21.15.6:0/3283342384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c1015f0 0x7ff16c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 -- 172.21.15.6:0/3283342384 >> 172.21.15.6:0/3283342384 conn(0x7ff16c077500 msgr2=0x7ff16c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 -- 172.21.15.6:0/3283342384 shutdown_connections 2023-11-17T00:16:47.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.540+0000 7ff1711ff700 1 -- 172.21.15.6:0/3283342384 wait complete. 2023-11-17T00:16:47.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.541+0000 7ff1711ff700 1 Processor -- start 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 -- start start 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 0x7ff16c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c105560 0x7ff16c1a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c073bd0 con 0x7ff16c100c50 2023-11-17T00:16:47.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a4a80 con 0x7ff16c1015f0 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.542+0000 7ff1711ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a4bc0 con 0x7ff16c105560 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59744/0 (socket says 172.21.15.6:59744) 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 -- 172.21.15.6:0/4115667926 learned_addr learned my addr 172.21.15.6:0/4115667926 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16b59e700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c105560 0x7ff16c1a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff163fff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 0x7ff16c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 -- 172.21.15.6:0/4115667926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c105560 msgr2=0x7ff16c1a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c105560 0x7ff16c1a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 -- 172.21.15.6:0/4115667926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 msgr2=0x7ff16c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 0x7ff16c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 -- 172.21.15.6:0/4115667926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0095f0 con 0x7ff16c100c50 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff163fff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 0x7ff16c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.543+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00f690 tx=0x7ff15c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.544+0000 7ff168d99700 1 -- 172.21.15.6:0/4115667926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c01d070 con 0x7ff16c100c50 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.544+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c110360 con 0x7ff16c100c50 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.544+0000 7ff168d99700 1 -- 172.21.15.6:0/4115667926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15c005050 con 0x7ff16c100c50 2023-11-17T00:16:47.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.544+0000 7ff168d99700 1 -- 172.21.15.6:0/4115667926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c004a20 con 0x7ff16c100c50 2023-11-17T00:16:47.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.544+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1101d0 con 0x7ff16c100c50 2023-11-17T00:16:47.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.547+0000 7ff168d99700 1 -- 172.21.15.6:0/4115667926 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff15c0051c0 con 0x7ff16c100c50 2023-11-17T00:16:47.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff184c18b80 1 Processor -- start 2023-11-17T00:16:47.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff184c18b80 1 -- start start 2023-11-17T00:16:47.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff168d99700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 0x7ff1580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff168d99700 1 -- 172.21.15.6:0/4115667926 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff15c0cded0 con 0x7ff16c100c50 2023-11-17T00:16:47.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 0x55b5485cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.548+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff184c18b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5484d50e0 con 0x55b5485d04f0 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff184c18b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b548527920 con 0x55b5485d4930 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff184c18b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b548520db0 con 0x55b5485cfbe0 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff16b59e700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 0x7ff1580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59756/0 (socket says 172.21.15.6:59756) 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 -- 172.21.15.6:0/1032727260 learned_addr learned my addr 172.21.15.6:0/1032727260 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 0x55b5485cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff16b59e700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 -- 172.21.15.6:0/1032727260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 msgr2=0x55b5485cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 0x55b5485cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 -- 172.21.15.6:0/1032727260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 msgr2=0x55b5485d4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff163fff700 1 -- 172.21.15.6:0/1032727260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5483a26b0 con 0x55b5485d04f0 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.549+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 0x55b5485cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:47.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.550+0000 7ff16b59e700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 0x7ff1580a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff164009ec0 tx=0x7ff16400fc40 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.550+0000 7ff163fff700 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff154009cc0 tx=0x7ff1540113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca6a7358fbe5d3f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.550+0000 7ff1617fa700 1 -- 172.21.15.6:0/1032727260 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011bb0 con 0x55b5485d04f0 2023-11-17T00:16:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.550+0000 7ff1617fa700 1 -- 172.21.15.6:0/1032727260 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff154011d10 con 0x55b5485d04f0 2023-11-17T00:16:47.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.550+0000 7ff1617fa700 1 -- 172.21.15.6:0/1032727260 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400f6a0 con 0x55b5485d04f0 2023-11-17T00:16:47.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 -- 172.21.15.6:0/1032727260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 msgr2=0x55b5485d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff154009cc0 tx=0x7ff1540113e0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 -- 172.21.15.6:0/1032727260 shutdown_connections 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485cfbe0 0x55b5485cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1032727260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485d43f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 -- 172.21.15.6:0/1032727260 >> 172.21.15.6:0/1032727260 conn(0x55b5485c53e0 msgr2=0x55b5485cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 -- 172.21.15.6:0/1032727260 shutdown_connections 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.551+0000 7ff184c18b80 1 -- 172.21.15.6:0/1032727260 wait complete. 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.552+0000 7ff184c18b80 1 -- start start 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.552+0000 7ff184c18b80 1 Processor -- start 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 0x55b5485d9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5484c2960 con 0x55b5485cfbe0 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5484c2600 con 0x55b5485d4930 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5483a49a0 con 0x55b5485d04f0 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff184c18b80 10 client.0 _ll_register_callbacks cb 0x55b5484e0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59772/0 (socket says 172.21.15.6:59772) 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 -- 172.21.15.6:0/3790475902 learned_addr learned my addr 172.21.15.6:0/3790475902 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16b59e700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff163fff700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 0x55b5485d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 -- 172.21.15.6:0/3790475902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 msgr2=0x55b5485d9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 0x55b5485d9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 -- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 msgr2=0x55b5485d9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff16ad9d700 1 -- 172.21.15.6:0/3790475902 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1540098f0 con 0x55b5485cfbe0 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.553+0000 7ff163fff700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 0x55b5485d9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:47.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff15c09abc0 tx=0x7ff15c09ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daec57a7392f7df8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c004df0 con 0x55b5485cfbe0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff15c0b3200 con 0x55b5485cfbe0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5485c2ba0 con 0x55b5485cfbe0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff15c0b33c0 con 0x55b5485cfbe0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff184c18b80 10 client.16641 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b5484e7210 con 0x55b5485cfbe0 2023-11-17T00:16:47.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15c0ca080 con 0x55b5485cfbe0 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.554+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff15c0a2b60 con 0x55b5485cfbe0 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 20 client.16641 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b5485e93f0 0x55b5485eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 4 client.16641 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5484e3120 con 0x55b5485e93f0 2023-11-17T00:16:47.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.555+0000 7ff163fff700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b5485e93f0 0x55b5485eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.556+0000 7ff163fff700 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b5485e93f0 0x55b5485eb7d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a74f3fedec01d4 server_cookie=d3d4f553b51c500f in_seq=0 out_seq=0 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.556+0000 7ff1527fc700 10 client.16641 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.556+0000 7ff1527fc700 1 -- 172.21.15.6:0/3790475902 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b5484e3120 con 0x55b5485e93f0 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.556+0000 7ff1527fc700 10 client.16641 handle_command_reply: tid=0 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.556+0000 7ff184c18b80 1 client.16641 shutdown 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 msgr2=0x55b5485dbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff15c09abc0 tx=0x7ff15c09ad70 comp rx=0 tx=0).stop 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 shutdown_connections 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b5485e93f0 0x55b5485eb7d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485cfbe0 0x55b5485dbfb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485d4930 0x55b5485d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff1527fc700 0 client.16641 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 --2- 172.21.15.6:0/3790475902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d04f0 0x55b5485d9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 >> 172.21.15.6:0/3790475902 conn(0x55b5485c53e0 msgr2=0x55b5485c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 shutdown_connections 2023-11-17T00:16:47.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 1 -- 172.21.15.6:0/3790475902 wait complete. 2023-11-17T00:16:47.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.557+0000 7ff184c18b80 20 client.16641 trim_cache size 0 max 16384 2023-11-17T00:16:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 Processor -- start 2023-11-17T00:16:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 -- start start 2023-11-17T00:16:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5485c4f60 con 0x55b5485d04f0 2023-11-17T00:16:47.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5485d3370 con 0x55b5485cfbe0 2023-11-17T00:16:47.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff184c18b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5485e0f90 con 0x55b5485d4930 2023-11-17T00:16:47.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff163fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42002/0 (socket says 172.21.15.6:42002) 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 -- 172.21.15.6:0/1914611914 learned_addr learned my addr 172.21.15.6:0/1914611914 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.575+0000 7ff163fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59776/0 (socket says 172.21.15.6:59776) 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16b59e700 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 -- 172.21.15.6:0/1914611914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 msgr2=0x55b5485dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 -- 172.21.15.6:0/1914611914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 msgr2=0x55b5485cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 -- 172.21.15.6:0/1914611914 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5484e6f60 con 0x55b5485cfbe0 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16b59e700 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0976a0 tx=0x7ff15c0b6f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e747d02f9a4e3b9a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff161ffb700 1 -- 172.21.15.6:0/1914611914 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0cb0f0 con 0x55b5485cfbe0 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff161ffb700 1 -- 172.21.15.6:0/1914611914 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5485c2ba0 con 0x55b5485cfbe0 2023-11-17T00:16:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.576+0000 7ff161ffb700 1 -- 172.21.15.6:0/1914611914 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0a2b60 con 0x55b5485cfbe0 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 -- 172.21.15.6:0/1914611914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 msgr2=0x55b5485e0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0976a0 tx=0x7ff15c0b6f60 comp rx=0 tx=0).stop 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 -- 172.21.15.6:0/1914611914 shutdown_connections 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485e0a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 --2- 172.21.15.6:0/1914611914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 -- 172.21.15.6:0/1914611914 >> 172.21.15.6:0/1914611914 conn(0x55b5485c53e0 msgr2=0x55b5485ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 -- 172.21.15.6:0/1914611914 shutdown_connections 2023-11-17T00:16:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.577+0000 7ff184c18b80 1 -- 172.21.15.6:0/1914611914 wait complete. 2023-11-17T00:16:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 -- start start 2023-11-17T00:16:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 Processor -- start 2023-11-17T00:16:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54863d280 con 0x55b5485d04f0 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54863d3c0 con 0x55b5485cfbe0 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54863d550 con 0x55b5485d4930 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.578+0000 7ff184c18b80 10 client.0 _ll_register_callbacks cb 0x7ff15c09a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42008/0 (socket says 172.21.15.6:42008) 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16ad9d700 1 -- 172.21.15.6:0/4057892154 learned_addr learned my addr 172.21.15.6:0/4057892154 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16b59e700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 -- 172.21.15.6:0/4057892154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 msgr2=0x55b5485ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485ddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 -- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 msgr2=0x55b5485dd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 -- 172.21.15.6:0/4057892154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5484e7210 con 0x55b5485d04f0 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16b59e700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:47.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff163fff700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485dd5b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff154009c90 tx=0x7ff154011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c680e7775b838e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff1527fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.579+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15402c040 con 0x55b5485d04f0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff154012b40 con 0x55b5485d04f0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff154011d10 con 0x55b5485d04f0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff154005040 con 0x55b5485d04f0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 10 client.16647 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b548337ce0 con 0x55b5485d04f0 2023-11-17T00:16:47.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff154011d10 con 0x55b5485d04f0 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff154009cf0 con 0x55b5485d04f0 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 20 client.16647 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b54863ebe0 0x55b548640ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 4 client.16647 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.580+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b5484ef6e0 con 0x55b54863ebe0 2023-11-17T00:16:47.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.581+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b54863ebe0 0x55b548640ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.581+0000 7ff16ad9d700 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b54863ebe0 0x55b548640ff0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fd89ffe0e998f7 server_cookie=c70a74f2e2e14240 in_seq=0 out_seq=0 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.581+0000 7ff1527fc700 10 client.16647 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.582+0000 7ff1527fc700 1 -- 172.21.15.6:0/4057892154 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b5484ef6e0 con 0x55b54863ebe0 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.582+0000 7ff1527fc700 10 client.16647 handle_command_reply: tid=0 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.582+0000 7ff184c18b80 1 client.16647 shutdown 2023-11-17T00:16:47.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 msgr2=0x55b5485dd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485dd5b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff154009c90 tx=0x7ff154011ad0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 shutdown_connections 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b5485d4930 0x55b5485ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b5485cfbe0 0x55b5485dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55b54863ebe0 0x55b548640ff0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 --2- 172.21.15.6:0/4057892154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55b5485d04f0 0x55b5485dd5b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 >> 172.21.15.6:0/4057892154 conn(0x55b5485c53e0 msgr2=0x55b5485d6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff1527fc700 0 client.16647 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 shutdown_connections 2023-11-17T00:16:47.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 1 -- 172.21.15.6:0/4057892154 wait complete. 2023-11-17T00:16:47.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.583+0000 7ff184c18b80 20 client.16647 trim_cache size 0 max 16384 2023-11-17T00:16:47.593 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:47.593 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:16:47.594 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "8366cbc0-0fbc-4d2d-a8eb-ed2b5256f79a", 2023-11-17T00:16:47.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:16:47.594 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 msgr2=0x7ff1580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 0x7ff1580a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff164009ec0 tx=0x7ff16400fc40 comp rx=0 tx=0).stop 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 msgr2=0x7ff16c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00f690 tx=0x7ff15c00bfd0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 shutdown_connections 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ff1580a12c0 0x7ff1580a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff16c100c50 0x7ff16c072df0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff16c105560 0x7ff16c1a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 --2- 172.21.15.6:0/4115667926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c1015f0 0x7ff16c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 >> 172.21.15.6:0/4115667926 conn(0x7ff16c077500 msgr2=0x7ff16c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 shutdown_connections 2023-11-17T00:16:47.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.584+0000 7ff1711ff700 1 -- 172.21.15.6:0/4115667926 wait complete. 2023-11-17T00:16:47.604 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:16:47.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 -- 172.21.15.6:0/3802189263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0100c50 msgr2=0x7fd9c0101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/3802189263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0100c50 0x7fd9c0101020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00099c0 tx=0x7fd9b0009cd0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 -- 172.21.15.6:0/3802189263 shutdown_connections 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/3802189263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0100c50 0x7fd9c0101020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/3802189263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c0105560 0x7fd9c010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/3802189263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c01015f0 0x7fd9c0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 -- 172.21.15.6:0/3802189263 >> 172.21.15.6:0/3802189263 conn(0x7fd9c00774f0 msgr2=0x7fd9c00778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 -- 172.21.15.6:0/3802189263 shutdown_connections 2023-11-17T00:16:47.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.811+0000 7fd9c61ff700 1 -- 172.21.15.6:0/3802189263 wait complete. 2023-11-17T00:16:47.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 Processor -- start 2023-11-17T00:16:47.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 -- start start 2023-11-17T00:16:47.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c01015f0 0x7fd9c0107660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 0x7fd9c0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a8ce0 con 0x7fd9c0105560 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.812+0000 7fd9c61ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a8e50 con 0x7fd9c01015f0 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9c61ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a9010 con 0x7fd9c0100c50 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 0x7fd9c0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41428/0 (socket says 172.21.15.6:41428) 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/2347606889 learned_addr learned my addr 172.21.15.6:0/2347606889 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9beffd700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c01015f0 0x7fd9c0107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/2347606889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c01015f0 msgr2=0x7fd9c0107660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c01015f0 0x7fd9c0107660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 msgr2=0x7fd9c0107ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 0x7fd9c0107ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/2347606889 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00095f0 con 0x7fd9c0100c50 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bffff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 0x7fd9c0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:47.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.813+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009990 tx=0x7fd9b000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.814+0000 7fd9bcff9700 1 -- 172.21.15.6:0/2347606889 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b001d070 con 0x7fd9c0100c50 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.814+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c0110360 con 0x7fd9c0100c50 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.814+0000 7fd9bcff9700 1 -- 172.21.15.6:0/2347606889 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b0005030 con 0x7fd9c0100c50 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.814+0000 7fd9bcff9700 1 -- 172.21.15.6:0/2347606889 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0004a50 con 0x7fd9c0100c50 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.814+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01101d0 con 0x7fd9c0100c50 2023-11-17T00:16:47.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.817+0000 7fd9bcff9700 1 -- 172.21.15.6:0/2347606889 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd9b0005430 con 0x7fd9c0100c50 2023-11-17T00:16:47.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.819+0000 7fd9d9b72b80 1 Processor -- start 2023-11-17T00:16:47.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.819+0000 7fd9d9b72b80 1 -- start start 2023-11-17T00:16:47.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.819+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.819+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.819+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d5a8060 con 0x55ed6d6a2b60 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d5fa8a0 con 0x55ed6d6a3470 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d5f3d30 con 0x55ed6d6a78b0 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bcff9700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 0x7fd9a80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42038/0 (socket says 172.21.15.6:42038) 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/3798729779 learned_addr learned my addr 172.21.15.6:0/3798729779 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bffff700 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bcff9700 1 -- 172.21.15.6:0/2347606889 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9b00cdd70 con 0x7fd9c0100c50 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bffff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 0x7fd9a80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/3798729779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 msgr2=0x55ed6d6a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/3798729779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 msgr2=0x55ed6d6a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/3798729779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6d475630 con 0x55ed6d6a3470 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.820+0000 7fd9beffd700 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.821+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0097540 tx=0x7fd9b006cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5cb7f83af162fd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.821+0000 7fd9bffff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 0x7fd9a80a37f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009550 tx=0x7fd9b8009130 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.821+0000 7fd9a5ffb700 1 -- 172.21.15.6:0/3798729779 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b009aad0 con 0x55ed6d6a3470 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.821+0000 7fd9a5ffb700 1 -- 172.21.15.6:0/3798729779 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b009eb00 con 0x55ed6d6a3470 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.821+0000 7fd9a5ffb700 1 -- 172.21.15.6:0/3798729779 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0066690 con 0x55ed6d6a3470 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/3798729779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 msgr2=0x55ed6d6a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0097540 tx=0x7fd9b006cb40 comp rx=0 tx=0).stop 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/3798729779 shutdown_connections 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/3798729779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6a7370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/3798729779 >> 172.21.15.6:0/3798729779 conn(0x55ed6d698360 msgr2=0x55ed6d69db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/3798729779 shutdown_connections 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/3798729779 wait complete. 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.822+0000 7fd9d9b72b80 1 -- start start 2023-11-17T00:16:47.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 Processor -- start 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d5958e0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d595580 con 0x55ed6d6a3470 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d477920 con 0x55ed6d6a2b60 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9d9b72b80 10 client.0 _ll_register_callbacks cb 0x55ed6d5b3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:47.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59810/0 (socket says 172.21.15.6:59810) 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9bffff700 1 -- 172.21.15.6:0/1177507793 learned_addr learned my addr 172.21.15.6:0/1177507793 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9beffd700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.823+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 -- 172.21.15.6:0/1177507793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a2b60 msgr2=0x55ed6d6aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 -- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 msgr2=0x55ed6d6ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 -- 172.21.15.6:0/1177507793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01101d0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9bffff700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8019ac0 tx=0x7fd9b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b3e3e033c75e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd99effd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8022070 con 0x55ed6d6a78b0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed6d695b20 con 0x55ed6d6a78b0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.824+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9b8004bb0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8004d70 con 0x55ed6d6a78b0 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 10 client.16659 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ed6d5ba190 con 0x55ed6d6a78b0 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed6d695b20 con 0x55ed6d6a78b0 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9b801b680 con 0x55ed6d6a78b0 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 10 client.16659 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 20 client.16659 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:47.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d6bc500 0x55ed6d6be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 4 client.16659 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:16:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.825+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed6d5b60a0 con 0x55ed6d6bc500 2023-11-17T00:16:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.826+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d6bc500 0x55ed6d6be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.826+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d6bc500 0x55ed6d6be910 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13f9f494aea7158 server_cookie=41c5653e85df89a1 in_seq=0 out_seq=0 2023-11-17T00:16:47.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.826+0000 7fd99effd700 10 client.16659 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd99effd700 1 -- 172.21.15.6:0/1177507793 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed6d5b60a0 con 0x55ed6d6bc500 2023-11-17T00:16:47.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd99effd700 10 client.16659 handle_command_reply: tid=0 2023-11-17T00:16:47.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 client.16659 shutdown 2023-11-17T00:16:47.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 msgr2=0x55ed6d6ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8019ac0 tx=0x7fd9b8004ad0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 shutdown_connections 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d6bc500 0x55ed6d6be910 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6ac700 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd99effd700 0 client.16659 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1177507793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 >> 172.21.15.6:0/1177507793 conn(0x55ed6d698360 msgr2=0x55ed6d6a8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 shutdown_connections 2023-11-17T00:16:47.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.827+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1177507793 wait complete. 2023-11-17T00:16:47.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.828+0000 7fd9d9b72b80 20 client.16659 trim_cache size 0 max 16384 2023-11-17T00:16:47.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.845+0000 7fd9d9b72b80 1 Processor -- start 2023-11-17T00:16:47.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.845+0000 7fd9d9b72b80 1 -- start start 2023-11-17T00:16:47.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.845+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.845+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d6a3c00 con 0x55ed6d6a2b60 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d6acd00 con 0x55ed6d6a3470 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d695660 con 0x55ed6d6a78b0 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59814/0 (socket says 172.21.15.6:59814) 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/1585815702 learned_addr learned my addr 172.21.15.6:0/1585815702 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bffff700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9beffd700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/1585815702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 msgr2=0x55ed6d6aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/1585815702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 msgr2=0x55ed6d6ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/1585815702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6d6a9f60 con 0x55ed6d6a2b60 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9beffd700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.846+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0066a30 tx=0x7fd9b001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892b45104adb7ede server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9a67fc700 1 -- 172.21.15.6:0/1585815702 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b009aad0 con 0x55ed6d6a2b60 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9a67fc700 1 -- 172.21.15.6:0/1585815702 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b00051a0 con 0x55ed6d6a2b60 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9a67fc700 1 -- 172.21.15.6:0/1585815702 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b009af90 con 0x55ed6d6a2b60 2023-11-17T00:16:47.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1585815702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 msgr2=0x55ed6d6ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0066a30 tx=0x7fd9b001a040 comp rx=0 tx=0).stop 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1585815702 shutdown_connections 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a2b60 0x55ed6d6ab9d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/1585815702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a3470 0x55ed6d6ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1585815702 >> 172.21.15.6:0/1585815702 conn(0x55ed6d698360 msgr2=0x55ed6d6ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1585815702 shutdown_connections 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.847+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/1585815702 wait complete. 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 -- start start 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 Processor -- start 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a3470 0x55ed6d709c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d6b1ae0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d6b1c50 con 0x55ed6d6a2b60 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6d6b1e10 con 0x55ed6d6a3470 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.848+0000 7fd9d9b72b80 10 client.0 _ll_register_callbacks cb 0x55ed6d5b3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:42068/0 (socket says 172.21.15.6:42068) 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bf7fe700 1 -- 172.21.15.6:0/396934079 learned_addr learned my addr 172.21.15.6:0/396934079 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:16:47.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9beffd700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a3470 0x55ed6d709c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 -- 172.21.15.6:0/396934079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a3470 msgr2=0x55ed6d709c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a3470 0x55ed6d709c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 -- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 msgr2=0x55ed6d709750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:16:47.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 -- 172.21.15.6:0/396934079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed6d475630 con 0x55ed6d6a78b0 2023-11-17T00:16:47.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:16:47.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd9bffff700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6b1500 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007830 tx=0x7fd9b801bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160a2bf693703103 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:16:47.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.849+0000 7fd99effd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:16:47.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8022040 con 0x55ed6d6a78b0 2023-11-17T00:16:47.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9b009eb00 con 0x55ed6d6a78b0 2023-11-17T00:16:47.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9b8004b40 con 0x55ed6d6a78b0 2023-11-17T00:16:47.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8007d40 con 0x55ed6d6a78b0 2023-11-17T00:16:47.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 10 client.16665 fetch_fsmap learned FSMap version 22 2023-11-17T00:16:47.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ed6d421de0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b009eb00 con 0x55ed6d6a78b0 2023-11-17T00:16:47.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9b80043a0 con 0x55ed6d6a78b0 2023-11-17T00:16:47.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 10 client.16665 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:16:47.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:16:47.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 20 client.16665 populate_metadata read hostname 'smithi006' 2023-11-17T00:16:47.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d711330 0x55ed6d713740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:16:47.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 4 client.16665 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:16:47.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.850+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed6d5c2660 con 0x55ed6d711330 2023-11-17T00:16:47.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.851+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d711330 0x55ed6d713740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:16:47.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.851+0000 7fd9bf7fe700 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d711330 0x55ed6d713740 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ec3d5edbb2b61e server_cookie=632816a1ad2cdd23 in_seq=0 out_seq=0 2023-11-17T00:16:47.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.851+0000 7fd99effd700 10 client.16665 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd99effd700 1 -- 172.21.15.6:0/396934079 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ed6d5c2660 con 0x55ed6d711330 2023-11-17T00:16:47.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd99effd700 10 client.16665 handle_command_reply: tid=0 2023-11-17T00:16:47.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 client.16665 shutdown 2023-11-17T00:16:47.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 msgr2=0x55ed6d6b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6b1500 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007830 tx=0x7fd9b801bde0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 shutdown_connections 2023-11-17T00:16:47.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55ed6d711330 0x55ed6d713740 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ed6d6a78b0 0x55ed6d6b1500 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ed6d6a2b60 0x55ed6d709750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 --2- 172.21.15.6:0/396934079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed6d6a3470 0x55ed6d709c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 >> 172.21.15.6:0/396934079 conn(0x55ed6d698360 msgr2=0x55ed6d6b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd99effd700 0 client.16665 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:16:47.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 shutdown_connections 2023-11-17T00:16:47.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 1 -- 172.21.15.6:0/396934079 wait complete. 2023-11-17T00:16:47.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.852+0000 7fd9d9b72b80 20 client.16665 trim_cache size 0 max 16384 2023-11-17T00:16:47.874 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:16:47.874 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:16:47.874 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:16:47.875 INFO:teuthology.orchestra.run.smithi006.stdout: "8366cbc0-0fbc-4d2d-a8eb-ed2b5256f79a": { 2023-11-17T00:16:47.875 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:16:47.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "8366cbc0-0fbc-4d2d-a8eb-ed2b5256f79a", 2023-11-17T00:16:47.876 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:16:47.876 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:16:47.876 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:16:47.876 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:16:47.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 msgr2=0x7fd9a80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 0x7fd9a80a37f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009550 tx=0x7fd9b8009130 comp rx=0 tx=0).stop 2023-11-17T00:16:47.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 msgr2=0x7fd9c0107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:16:47.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009990 tx=0x7fd9b000f690 comp rx=0 tx=0).stop 2023-11-17T00:16:47.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 shutdown_connections 2023-11-17T00:16:47.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd9a80a1340 0x7fd9a80a37f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9c0105560 0x7fd9c0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c0100c50 0x7fd9c0107120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 --2- 172.21.15.6:0/2347606889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9c01015f0 0x7fd9c0107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 >> 172.21.15.6:0/2347606889 conn(0x7fd9c00774f0 msgr2=0x7fd9c00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 shutdown_connections 2023-11-17T00:16:47.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:16:47.853+0000 7fd9c61ff700 1 -- 172.21.15.6:0/2347606889 wait complete. 2023-11-17T00:16:47.880 INFO:tasks.cephfs.filesystem:scrub status for tag:8366cbc0-0fbc-4d2d-a8eb-ed2b5256f79a - {'path': '/', 'tag': '8366cbc0-0fbc-4d2d-a8eb-ed2b5256f79a', 'options': 'recursive,force'} 2023-11-17T00:16:47.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:47 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:16:47.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:47 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:47.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:47 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:16:47.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:47 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:48.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:47 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:16:48.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:47 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:48.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:47 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:16:48.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:47 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:48.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:47 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:16:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:47 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:47 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:16:48.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:47 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:16:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:48 smithi006 ceph-mon[97840]: pgmap v244: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-11-17T00:16:49.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:48 smithi087 ceph-mon[131182]: pgmap v244: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-11-17T00:16:49.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:48 smithi186 ceph-mon[131487]: pgmap v244: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-11-17T00:16:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:50 smithi006 ceph-mon[97840]: pgmap v245: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 6 op/s 2023-11-17T00:16:51.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:50 smithi087 ceph-mon[131182]: pgmap v245: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 6 op/s 2023-11-17T00:16:51.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:50 smithi186 ceph-mon[131487]: pgmap v245: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 6 op/s 2023-11-17T00:16:52.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:51 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:16:52.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:51 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:16:52.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:51 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:16:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:52 smithi006 ceph-mon[97840]: pgmap v246: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:53.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:52 smithi087 ceph-mon[131182]: pgmap v246: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:53.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:52 smithi186 ceph-mon[131487]: pgmap v246: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-17T00:16:55.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:54 smithi006 ceph-mon[97840]: pgmap v247: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:00.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:56 smithi006 ceph-mon[97840]: pgmap v248: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:17:00.558 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:16:58 smithi006 ceph-mon[97840]: pgmap v249: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:54 smithi186 ceph-mon[131487]: pgmap v247: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:56 smithi186 ceph-mon[131487]: pgmap v248: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:16:58 smithi186 ceph-mon[131487]: pgmap v249: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:54 smithi087 ceph-mon[131182]: pgmap v247: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:56 smithi087 ceph-mon[131182]: pgmap v248: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2023-11-17T00:17:00.559 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:16:58 smithi087 ceph-mon[131182]: pgmap v249: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:01.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:00 smithi006 ceph-mon[97840]: pgmap v250: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:01.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:00 smithi087 ceph-mon[131182]: pgmap v250: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:01.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:00 smithi186 ceph-mon[131487]: pgmap v250: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2023-11-17T00:17:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:02 smithi006 ceph-mon[97840]: pgmap v251: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-17T00:17:03.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:02 smithi087 ceph-mon[131182]: pgmap v251: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-17T00:17:03.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:02 smithi186 ceph-mon[131487]: pgmap v251: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 16 op/s 2023-11-17T00:17:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:04 smithi006 ceph-mon[97840]: pgmap v252: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-17T00:17:05.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:04 smithi087 ceph-mon[131182]: pgmap v252: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-17T00:17:05.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:04 smithi186 ceph-mon[131487]: pgmap v252: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-17T00:17:07.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:06 smithi006 ceph-mon[97840]: pgmap v253: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:17:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:06 smithi087 ceph-mon[131182]: pgmap v253: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:17:07.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:06 smithi186 ceph-mon[131487]: pgmap v253: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:17:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:07 smithi006 ceph-mon[97840]: osdmap e98: 12 total, 12 up, 12 in 2023-11-17T00:17:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:07 smithi087 ceph-mon[131182]: osdmap e98: 12 total, 12 up, 12 in 2023-11-17T00:17:08.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:07 smithi186 ceph-mon[131487]: osdmap e98: 12 total, 12 up, 12 in 2023-11-17T00:17:09.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:08 smithi006 ceph-mon[97840]: pgmap v255: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:09.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:08 smithi087 ceph-mon[131182]: pgmap v255: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:09.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:08 smithi186 ceph-mon[131487]: pgmap v255: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:10 smithi006 ceph-mon[97840]: pgmap v256: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:10 smithi087 ceph-mon[131182]: pgmap v256: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:10 smithi186 ceph-mon[131487]: pgmap v256: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:17:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:12 smithi006 ceph-mon[97840]: pgmap v257: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:12 smithi087 ceph-mon[131182]: pgmap v257: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:13.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:12 smithi186 ceph-mon[131487]: pgmap v257: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:14 smithi087 ceph-mon[131182]: pgmap v258: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:14.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:14 smithi087 ceph-mon[131182]: osdmap e99: 12 total, 12 up, 12 in 2023-11-17T00:17:14.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:14 smithi186 ceph-mon[131487]: pgmap v258: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:14.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:14 smithi186 ceph-mon[131487]: osdmap e99: 12 total, 12 up, 12 in 2023-11-17T00:17:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:14 smithi006 ceph-mon[97840]: pgmap v258: 129 pgs: 129 active+clean; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:17:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:14 smithi006 ceph-mon[97840]: osdmap e99: 12 total, 12 up, 12 in 2023-11-17T00:17:16.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:16 smithi087 ceph-mon[131182]: pgmap v260: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 238 B/s wr, 2 op/s 2023-11-17T00:17:16.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:16 smithi186 ceph-mon[131487]: pgmap v260: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 238 B/s wr, 2 op/s 2023-11-17T00:17:16.978 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:16 smithi006 ceph-mon[97840]: pgmap v260: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 238 B/s wr, 2 op/s 2023-11-17T00:17:17.881 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:17:18.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 -- 172.21.15.6:0/503909884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc105560 msgr2=0x7f1abc10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/503909884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc105560 0x7f1abc10d130 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b0d0 tx=0x7f1ab400b3e0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 -- 172.21.15.6:0/503909884 shutdown_connections 2023-11-17T00:17:18.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/503909884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc105560 0x7f1abc10d130 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/503909884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc100c50 0x7f1abc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/503909884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc1015f0 0x7f1abc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.098+0000 7f1ac1bff700 1 -- 172.21.15.6:0/503909884 >> 172.21.15.6:0/503909884 conn(0x7f1abc077500 msgr2=0x7f1abc077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.099+0000 7f1ac1bff700 1 -- 172.21.15.6:0/503909884 shutdown_connections 2023-11-17T00:17:18.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.099+0000 7f1ac1bff700 1 -- 172.21.15.6:0/503909884 wait complete. 2023-11-17T00:17:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 Processor -- start 2023-11-17T00:17:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 -- start start 2023-11-17T00:17:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc1015f0 0x7f1abc198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc105560 0x7f1abc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19d650 con 0x7f1abc100c50 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19d7c0 con 0x7f1abc1015f0 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1ac1bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19d980 con 0x7f1abc105560 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45826/0 (socket says 172.21.15.6:45826) 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1abb7fe700 1 -- 172.21.15.6:0/3749229800 learned_addr learned my addr 172.21.15.6:0/3749229800 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.100+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc1015f0 0x7f1abc198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc105560 0x7f1abc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 -- 172.21.15.6:0/3749229800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc105560 msgr2=0x7f1abc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc105560 0x7f1abc19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 -- 172.21.15.6:0/3749229800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc1015f0 msgr2=0x7f1abc198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc1015f0 0x7f1abc198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 -- 172.21.15.6:0/3749229800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009d00 con 0x7f1abc100c50 2023-11-17T00:17:18.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9a0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1ab8ff9700 1 -- 172.21.15.6:0/3749229800 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009790 con 0x7f1abc100c50 2023-11-17T00:17:18.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1ab8ff9700 1 -- 172.21.15.6:0/3749229800 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aac010430 con 0x7f1abc100c50 2023-11-17T00:17:18.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc110310 con 0x7f1abc100c50 2023-11-17T00:17:18.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1ab8ff9700 1 -- 172.21.15.6:0/3749229800 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac018680 con 0x7f1abc100c50 2023-11-17T00:17:18.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.101+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc110180 con 0x7f1abc100c50 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.105+0000 7f1ab8ff9700 1 -- 172.21.15.6:0/3749229800 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1aac0105a0 con 0x7f1abc100c50 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.106+0000 7f1ab8ff9700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 0x7f1aa40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.106+0000 7f1ad5443b80 1 Processor -- start 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.106+0000 7f1ab8ff9700 1 -- 172.21.15.6:0/3749229800 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1aac014070 con 0x7f1abc100c50 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.106+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 0x7f1aa40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.106+0000 7f1ad5443b80 1 -- start start 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb501f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb506370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb407060 con 0x561deb5068b0 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb4598a0 con 0x561deb502470 2023-11-17T00:17:18.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb452d30 con 0x561deb501b60 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 0x7f1aa40a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1abc199c80 tx=0x7f1ab0009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb501f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45840/0 (socket says 172.21.15.6:45840) 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 -- 172.21.15.6:0/3305503603 learned_addr learned my addr 172.21.15.6:0/3305503603 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb506370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 -- 172.21.15.6:0/3305503603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 msgr2=0x561deb501f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb501f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 -- 172.21.15.6:0/3305503603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 msgr2=0x561deb506370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb506370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abbfff700 1 -- 172.21.15.6:0/3305503603 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561deb2d4630 con 0x561deb5068b0 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.107+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb506370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.108+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009c90 tx=0x7f1ab4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ebc82a1d981186 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.108+0000 7f1aa1ffb700 1 -- 172.21.15.6:0/3305503603 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e040 con 0x561deb5068b0 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.108+0000 7f1aa1ffb700 1 -- 172.21.15.6:0/3305503603 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab40093e0 con 0x561deb5068b0 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.108+0000 7f1aa1ffb700 1 -- 172.21.15.6:0/3305503603 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4005070 con 0x561deb5068b0 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3305503603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 msgr2=0x561deb506d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009c90 tx=0x7f1ab4005a20 comp rx=0 tx=0).stop 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3305503603 shutdown_connections 2023-11-17T00:17:18.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb506d00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb501f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3305503603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb506370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3305503603 >> 172.21.15.6:0/3305503603 conn(0x561deb4f7360 msgr2=0x561deb4fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3305503603 shutdown_connections 2023-11-17T00:17:18.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.109+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3305503603 wait complete. 2023-11-17T00:17:18.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 -- start start 2023-11-17T00:17:18.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 Processor -- start 2023-11-17T00:17:18.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb502470 0x561deb50b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 0x561deb50b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb3f48e0 con 0x561deb502470 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb3f4580 con 0x561deb501b60 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb2d6920 con 0x561deb5068b0 2023-11-17T00:17:18.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.110+0000 7f1ad5443b80 10 client.0 _ll_register_callbacks cb 0x561deb412780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:18.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:50862/0 (socket says 172.21.15.6:50862) 2023-11-17T00:17:18.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 -- 172.21.15.6:0/4116330210 learned_addr learned my addr 172.21.15.6:0/4116330210 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb502470 0x561deb50b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abbfff700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 0x561deb50b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 -- 172.21.15.6:0/4116330210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 msgr2=0x561deb50b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 0x561deb50b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 -- 172.21.15.6:0/4116330210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb502470 msgr2=0x561deb50b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb502470 0x561deb50b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 -- 172.21.15.6:0/4116330210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc10d940 con 0x561deb501b60 2023-11-17T00:17:18.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abbfff700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 0x561deb50b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:18.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1aac096a10 tx=0x7f1aac099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11807f25d05d462 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1a9affd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:17:18.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.111+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0b1070 con 0x561deb501b60 2023-11-17T00:17:18.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1aac0a23f0 con 0x561deb501b60 2023-11-17T00:17:18.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561deb4f4b20 con 0x561deb501b60 2023-11-17T00:17:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac099490 con 0x561deb501b60 2023-11-17T00:17:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 10 client.26813 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561deb419190 con 0x561deb501b60 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561deb4f4b20 con 0x561deb501b60 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1aac0a1ff0 con 0x561deb501b60 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 10 client.26813 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:18.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 20 client.26813 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb51b900 0x561deb51dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 4 client.26813 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.112+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561deb4150a0 con 0x561deb51b900 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.113+0000 7f1abaffd700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb51b900 0x561deb51dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.113+0000 7f1abaffd700 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb51b900 0x561deb51dd10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e6aede6c2fef5 server_cookie=f659bb2ea0bcb5c5 in_seq=0 out_seq=0 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.113+0000 7f1a9affd700 10 client.26813 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1a9affd700 1 -- 172.21.15.6:0/4116330210 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561deb4150a0 con 0x561deb51b900 2023-11-17T00:17:18.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1a9affd700 10 client.26813 handle_command_reply: tid=0 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 client.26813 shutdown 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 msgr2=0x561deb50e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1aac096a10 tx=0x7f1aac099a30 comp rx=0 tx=0).stop 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 shutdown_connections 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb5068b0 0x561deb50b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb50e050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb51b900 0x561deb51dd10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/4116330210 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb502470 0x561deb50b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 >> 172.21.15.6:0/4116330210 conn(0x561deb4f7360 msgr2=0x561deb507d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1a9affd700 0 client.26813 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 shutdown_connections 2023-11-17T00:17:18.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.114+0000 7f1ad5443b80 1 -- 172.21.15.6:0/4116330210 wait complete. 2023-11-17T00:17:18.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.115+0000 7f1ad5443b80 20 client.26813 trim_cache size 0 max 16384 2023-11-17T00:17:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.132+0000 7f1ad5443b80 1 Processor -- start 2023-11-17T00:17:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.132+0000 7f1ad5443b80 1 -- start start 2023-11-17T00:17:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb4fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb517b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb51b720 con 0x561deb5068b0 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb4f4a10 con 0x561deb502470 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb50c290 con 0x561deb501b60 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45854/0 (socket says 172.21.15.6:45854) 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 -- 172.21.15.6:0/2419968045 learned_addr learned my addr 172.21.15.6:0/2419968045 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb4fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abaffd700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb517b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 -- 172.21.15.6:0/2419968045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 msgr2=0x561deb4fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb4fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 -- 172.21.15.6:0/2419968045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 msgr2=0x561deb517b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb517b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abbfff700 1 -- 172.21.15.6:0/2419968045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561deb4f8b00 con 0x561deb5068b0 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.133+0000 7f1abaffd700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb517b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1abbfff700 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009c90 tx=0x7f1ab4006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c614065a010d3faf server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1aa27fc700 1 -- 172.21.15.6:0/2419968045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e040 con 0x561deb5068b0 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1aa27fc700 1 -- 172.21.15.6:0/2419968045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab400bca0 con 0x561deb5068b0 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1aa27fc700 1 -- 172.21.15.6:0/2419968045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40052d0 con 0x561deb5068b0 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 -- 172.21.15.6:0/2419968045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 msgr2=0x561deb50dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009c90 tx=0x7f1ab4006000 comp rx=0 tx=0).stop 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 -- 172.21.15.6:0/2419968045 shutdown_connections 2023-11-17T00:17:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb50dbf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb501b60 0x561deb4fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/2419968045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb502470 0x561deb517b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.134+0000 7f1ad5443b80 1 -- 172.21.15.6:0/2419968045 >> 172.21.15.6:0/2419968045 conn(0x561deb4f7360 msgr2=0x561deb508c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.135+0000 7f1ad5443b80 1 -- 172.21.15.6:0/2419968045 shutdown_connections 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.135+0000 7f1ad5443b80 1 -- 172.21.15.6:0/2419968045 wait complete. 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.135+0000 7f1ad5443b80 1 -- start start 2023-11-17T00:17:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.135+0000 7f1ad5443b80 1 Processor -- start 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb5682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb502470 0x561deb5687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb512530 con 0x561deb5068b0 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb5126a0 con 0x561deb501b60 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561deb512860 con 0x561deb502470 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1ad5443b80 10 client.0 _ll_register_callbacks cb 0x7f1aac099960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45868/0 (socket says 172.21.15.6:45868) 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 -- 172.21.15.6:0/3935691189 learned_addr learned my addr 172.21.15.6:0/3935691189 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abaffd700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb502470 0x561deb5687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb5682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 -- 172.21.15.6:0/3935691189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb502470 msgr2=0x561deb5687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb502470 0x561deb5687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 -- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 msgr2=0x561deb5682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb5682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.136+0000 7f1abbfff700 1 -- 172.21.15.6:0/3935691189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561deb2d4630 con 0x561deb5068b0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1abbfff700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b7f0 tx=0x7f1ab4025b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aadd1cb489c420 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1a9affd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400be10 con 0x561deb5068b0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ab40093e0 con 0x561deb5068b0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1ab4007830 con 0x561deb5068b0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab40074a0 con 0x561deb5068b0 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1ad5443b80 10 client.16683 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561deb280de0 con 0x561deb5068b0 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.137+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab403f080 con 0x561deb5068b0 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ab400b100 con 0x561deb5068b0 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 10 client.16683 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 20 client.16683 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb56fc10 0x561deb571ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 4 client.16683 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561deb421660 con 0x561deb56fc10 2023-11-17T00:17:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb56fc10 0x561deb571ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1a9affd700 10 client.16683 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.138+0000 7f1abb7fe700 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb56fc10 0x561deb571ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fcb5c5381415f0 server_cookie=a8276c6b47e2d19a in_seq=0 out_seq=0 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1a9affd700 1 -- 172.21.15.6:0/3935691189 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561deb421660 con 0x561deb56fc10 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1a9affd700 10 client.16683 handle_command_reply: tid=0 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 client.16683 shutdown 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 msgr2=0x561deb511f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b7f0 tx=0x7f1ab4025b80 comp rx=0 tx=0).stop 2023-11-17T00:17:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 shutdown_connections 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x561deb56fc10 0x561deb571ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x561deb5068b0 0x561deb511f20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561deb501b60 0x561deb5682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 --2- 172.21.15.6:0/3935691189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561deb502470 0x561deb5687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 >> 172.21.15.6:0/3935691189 conn(0x561deb4f7360 msgr2=0x561deb5048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1a9affd700 0 client.16683 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 shutdown_connections 2023-11-17T00:17:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 1 -- 172.21.15.6:0/3935691189 wait complete. 2023-11-17T00:17:18.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.139+0000 7f1ad5443b80 20 client.16683 trim_cache size 0 max 16384 2023-11-17T00:17:18.161 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:18.161 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:17:18.161 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:17:18.161 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:18.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 msgr2=0x7f1aa40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 0x7f1aa40a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1abc199c80 tx=0x7f1ab0009260 comp rx=0 tx=0).stop 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 msgr2=0x7f1abc198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9a0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 shutdown_connections 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f1aa40a12c0 0x7f1aa40a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1abc100c50 0x7f1abc198670 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1abc105560 0x7f1abc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 --2- 172.21.15.6:0/3749229800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc1015f0 0x7f1abc198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 >> 172.21.15.6:0/3749229800 conn(0x7f1abc077500 msgr2=0x7f1abc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 shutdown_connections 2023-11-17T00:17:18.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.140+0000 7f1ac1bff700 1 -- 172.21.15.6:0/3749229800 wait complete. 2023-11-17T00:17:18.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:17:18.164 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:17:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 -- 172.21.15.6:0/124712475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9080746a0 msgr2=0x7fa908072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 --2- 172.21.15.6:0/124712475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9080746a0 0x7fa908072d20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b0a0 tx=0x7fa8f800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 -- 172.21.15.6:0/124712475 shutdown_connections 2023-11-17T00:17:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 --2- 172.21.15.6:0/124712475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa9080746a0 0x7fa908072d20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 --2- 172.21.15.6:0/124712475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 0x7fa90810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 --2- 172.21.15.6:0/124712475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa908074b00 0x7fa908073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 -- 172.21.15.6:0/124712475 >> 172.21.15.6:0/124712475 conn(0x7fa9080774d0 msgr2=0x7fa9080778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 -- 172.21.15.6:0/124712475 shutdown_connections 2023-11-17T00:17:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.369+0000 7fa90cbff700 1 -- 172.21.15.6:0/124712475 wait complete. 2023-11-17T00:17:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.370+0000 7fa90cbff700 1 Processor -- start 2023-11-17T00:17:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 -- start start 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 0x7fa90819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 0x7fa908198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908198c70 con 0x7fa908074b00 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908198de0 con 0x7fa9080746a0 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908198fa0 con 0x7fa90810cec0 2023-11-17T00:17:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:45880/0 (socket says 172.21.15.6:45880) 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa905d9b700 1 -- 172.21.15.6:0/67099940 learned_addr learned my addr 172.21.15.6:0/67099940 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa90659c700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 0x7fa90819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.371+0000 7fa906d9d700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 0x7fa908198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 -- 172.21.15.6:0/67099940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 msgr2=0x7fa908198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 0x7fa908198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 -- 172.21.15.6:0/67099940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 msgr2=0x7fa90819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 0x7fa90819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 -- 172.21.15.6:0/67099940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8009d00 con 0x7fa908074b00 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa90659c700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 0x7fa90819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa905d9b700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9a0 tx=0x7fa8fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc009790 con 0x7fa908074b00 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc010430 con 0x7fa908074b00 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9081125a0 con 0x7fa908074b00 2023-11-17T00:17:18.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc018680 con 0x7fa908074b00 2023-11-17T00:17:18.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.372+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908112410 con 0x7fa908074b00 2023-11-17T00:17:18.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.373+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908112730 con 0x7fa908074b00 2023-11-17T00:17:18.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.375+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa8fc0105a0 con 0x7fa908074b00 2023-11-17T00:17:18.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.377+0000 7fa8f77fe700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 0x7fa8f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.377+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa8fc014070 con 0x7fa908074b00 2023-11-17T00:17:18.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.377+0000 7fa90659c700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 0x7fa8f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.378+0000 7fa90659c700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 0x7fa8f00a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009fd0 tx=0x7fa8f8006210 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.381+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa8fc051730 con 0x7fa908074b00 2023-11-17T00:17:18.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.638+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa908107d20 con 0x7fa908074b00 2023-11-17T00:17:18.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.639+0000 7fa8f77fe700 1 -- 172.21.15.6:0/67099940 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fa8fc0962f0 con 0x7fa908074b00 2023-11-17T00:17:18.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:17:18.641 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 msgr2=0x7fa8f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 0x7fa8f00a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009fd0 tx=0x7fa8f8006210 comp rx=0 tx=0).stop 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 msgr2=0x7fa90819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9a0 tx=0x7fa8fc00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 shutdown_connections 2023-11-17T00:17:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa8f00a12c0 0x7fa8f00a3770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa90810cec0 0x7fa908198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080746a0 0x7fa90819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 --2- 172.21.15.6:0/67099940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa908074b00 0x7fa90819ead0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.641+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 >> 172.21.15.6:0/67099940 conn(0x7fa9080774d0 msgr2=0x7fa908104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.642+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 shutdown_connections 2023-11-17T00:17:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.642+0000 7fa90cbff700 1 -- 172.21.15.6:0/67099940 wait complete. 2023-11-17T00:17:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:17:18.663 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:17:18.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:18 smithi087 ceph-mon[131182]: pgmap v261: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:18.870 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:18 smithi186 ceph-mon[131487]: pgmap v261: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 -- 172.21.15.6:0/98714374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a68105560 msgr2=0x7f5a6810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/98714374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a68105560 0x7f5a6810d130 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000b0d0 tx=0x7f5a6000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 -- 172.21.15.6:0/98714374 shutdown_connections 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/98714374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a68105560 0x7f5a6810d130 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/98714374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 0x7f5a68101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/98714374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a681015f0 0x7f5a68104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 -- 172.21.15.6:0/98714374 >> 172.21.15.6:0/98714374 conn(0x7f5a680774f0 msgr2=0x7f5a680778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 -- 172.21.15.6:0/98714374 shutdown_connections 2023-11-17T00:17:18.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.871+0000 7f5a6dfff700 1 -- 172.21.15.6:0/98714374 wait complete. 2023-11-17T00:17:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.872+0000 7f5a6dfff700 1 Processor -- start 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.872+0000 7f5a6dfff700 1 -- start start 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 0x7f5a68196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a68105560 0x7f5a6819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819b500 con 0x7f5a681015f0 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819b670 con 0x7f5a68100c50 2023-11-17T00:17:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a6dfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819b830 con 0x7f5a68105560 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 0x7f5a68196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37702/0 (socket says 172.21.15.6:37702) 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a5ffff700 1 -- 172.21.15.6:0/1322733301 learned_addr learned my addr 172.21.15.6:0/1322733301 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.873+0000 7f5a67fff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a68105560 0x7f5a6819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 -- 172.21.15.6:0/1322733301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a68105560 msgr2=0x7f5a6819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a68105560 0x7f5a6819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 -- 172.21.15.6:0/1322733301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 msgr2=0x7f5a68196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 0x7f5a68196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 -- 172.21.15.6:0/1322733301 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60009d00 con 0x7f5a681015f0 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e940 tx=0x7f5a5000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a657fa700 1 -- 172.21.15.6:0/1322733301 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50009730 con 0x7f5a681015f0 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a657fa700 1 -- 172.21.15.6:0/1322733301 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a50010430 con 0x7f5a681015f0 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a68110360 con 0x7f5a681015f0 2023-11-17T00:17:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a657fa700 1 -- 172.21.15.6:0/1322733301 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5000cea0 con 0x7f5a681015f0 2023-11-17T00:17:18.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.874+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a681101d0 con 0x7f5a681015f0 2023-11-17T00:17:18.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.878+0000 7f5a657fa700 1 -- 172.21.15.6:0/1322733301 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5a50010810 con 0x7f5a681015f0 2023-11-17T00:17:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 Processor -- start 2023-11-17T00:17:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a657fa700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 0x7f5a540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 -- start start 2023-11-17T00:17:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a657fa700 1 -- 172.21.15.6:0/1322733301 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5a50014070 con 0x7f5a681015f0 2023-11-17T00:17:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a67fff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 0x7f5a540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b6d0060 con 0x56372b7cf8b0 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b7228a0 con 0x56372b7cab60 2023-11-17T00:17:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.879+0000 7f5a819abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b71bd30 con 0x56372b7cb470 2023-11-17T00:17:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33288/0 (socket says 172.21.15.6:33288) 2023-11-17T00:17:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 -- 172.21.15.6:0/3766241859 learned_addr learned my addr 172.21.15.6:0/3766241859 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a67fff700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 -- 172.21.15.6:0/3766241859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 msgr2=0x56372b7cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 -- 172.21.15.6:0/3766241859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 msgr2=0x56372b7cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 -- 172.21.15.6:0/3766241859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372b59d630 con 0x56372b7cab60 2023-11-17T00:17:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a677fe700 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009d80 tx=0x7f5a58011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8619798bcccee24 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.880+0000 7f5a67fff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 0x7f5a540a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a60006280 tx=0x7f5a60006210 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a5dffb700 1 -- 172.21.15.6:0/3766241859 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58004230 con 0x56372b7cab60 2023-11-17T00:17:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a5dffb700 1 -- 172.21.15.6:0/3766241859 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a58004390 con 0x56372b7cab60 2023-11-17T00:17:18.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a5dffb700 1 -- 172.21.15.6:0/3766241859 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58019860 con 0x56372b7cab60 2023-11-17T00:17:18.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 -- 172.21.15.6:0/3766241859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 msgr2=0x56372b7caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009d80 tx=0x7f5a58011720 comp rx=0 tx=0).stop 2023-11-17T00:17:18.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 -- 172.21.15.6:0/3766241859 shutdown_connections 2023-11-17T00:17:18.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7caf30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 --2- 172.21.15.6:0/3766241859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 -- 172.21.15.6:0/3766241859 >> 172.21.15.6:0/3766241859 conn(0x56372b7c0360 msgr2=0x56372b7c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.881+0000 7f5a819abb80 1 -- 172.21.15.6:0/3766241859 shutdown_connections 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.882+0000 7f5a819abb80 1 -- 172.21.15.6:0/3766241859 wait complete. 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.882+0000 7f5a819abb80 1 -- start start 2023-11-17T00:17:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.882+0000 7f5a819abb80 1 Processor -- start 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b6bd8e0 con 0x56372b7cf8b0 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b6bd580 con 0x56372b7cab60 2023-11-17T00:17:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b59f920 con 0x56372b7cb470 2023-11-17T00:17:18.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a819abb80 10 client.0 _ll_register_callbacks cb 0x56372b6db780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:18.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33296/0 (socket says 172.21.15.6:33296) 2023-11-17T00:17:18.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37728/0 (socket says 172.21.15.6:37728) 2023-11-17T00:17:18.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a677fe700 1 -- 172.21.15.6:0/77471 learned_addr learned my addr 172.21.15.6:0/77471 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a5ffff700 1 -- 172.21.15.6:0/77471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 msgr2=0x56372b7d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a5ffff700 1 -- 172.21.15.6:0/77471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 msgr2=0x56372b7d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.883+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a5ffff700 1 -- 172.21.15.6:0/77471 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a580099e0 con 0x56372b7cb470 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a677fe700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7d4220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a500ca590 tx=0x7f5a5009a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3406d8518aacf4b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:17:18.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50099600 con 0x56372b7cb470 2023-11-17T00:17:18.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56372b7bdb20 con 0x56372b7cb470 2023-11-17T00:17:18.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5a500a1ed0 con 0x56372b7cb470 2023-11-17T00:17:18.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a500a2090 con 0x56372b7cb470 2023-11-17T00:17:18.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a819abb80 10 client.26809 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:18.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56372b6e2190 con 0x56372b7cb470 2023-11-17T00:17:18.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.884+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56372b7bdb20 con 0x56372b7cb470 2023-11-17T00:17:18.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a50095d00 con 0x56372b7cb470 2023-11-17T00:17:18.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 10 client.26809 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 20 client.26809 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b7e4800 0x56372b7e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 4 client.26809 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56372b6de0a0 con 0x56372b7e4800 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.885+0000 7f5a677fe700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b7e4800 0x56372b7e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.886+0000 7f5a677fe700 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b7e4800 0x56372b7e6c10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78f8598e0dd673d server_cookie=e8ad5ad2f799facc in_seq=0 out_seq=0 2023-11-17T00:17:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.886+0000 7f5a46ffd700 10 client.26809 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.886+0000 7f5a46ffd700 1 -- 172.21.15.6:0/77471 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56372b6de0a0 con 0x56372b7e4800 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a46ffd700 10 client.26809 handle_command_reply: tid=0 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 client.26809 shutdown 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 msgr2=0x56372b7d4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7d4220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a500ca590 tx=0x7f5a5009a050 comp rx=0 tx=0).stop 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 shutdown_connections 2023-11-17T00:17:18.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b7e4800 0x56372b7e6c10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a46ffd700 0 client.26809 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 --2- 172.21.15.6:0/77471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7d4220 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 >> 172.21.15.6:0/77471 conn(0x56372b7c0360 msgr2=0x56372b7c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 shutdown_connections 2023-11-17T00:17:18.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 1 -- 172.21.15.6:0/77471 wait complete. 2023-11-17T00:17:18.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.887+0000 7f5a819abb80 20 client.26809 trim_cache size 0 max 16384 2023-11-17T00:17:18.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 Processor -- start 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 -- start start 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b7cbbc0 con 0x56372b7cf8b0 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b7bfee0 con 0x56372b7cab60 2023-11-17T00:17:18.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.901+0000 7f5a819abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b7e7270 con 0x56372b7cb470 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33298/0 (socket says 172.21.15.6:33298) 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37734/0 (socket says 172.21.15.6:37734) 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 -- 172.21.15.6:0/1453946470 learned_addr learned my addr 172.21.15.6:0/1453946470 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 -- 172.21.15.6:0/1453946470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 msgr2=0x56372b7cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 -- 172.21.15.6:0/1453946470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 msgr2=0x56372b7db9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 -- 172.21.15.6:0/1453946470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372b6e1ee0 con 0x56372b7cab60 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a67fff700 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a677fe700 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800a010 tx=0x7f5a58009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a1e761e63512a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a5e7fc700 1 -- 172.21.15.6:0/1453946470 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5802b560 con 0x56372b7cab60 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a5e7fc700 1 -- 172.21.15.6:0/1453946470 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a5802bd20 con 0x56372b7cab60 2023-11-17T00:17:18.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.902+0000 7f5a5e7fc700 1 -- 172.21.15.6:0/1453946470 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580056c0 con 0x56372b7cab60 2023-11-17T00:17:18.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 -- 172.21.15.6:0/1453946470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 msgr2=0x56372b7cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800a010 tx=0x7f5a58009260 comp rx=0 tx=0).stop 2023-11-17T00:17:18.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 -- 172.21.15.6:0/1453946470 shutdown_connections 2023-11-17T00:17:18.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cf8b0 0x56372b7db9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7cd8b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 --2- 172.21.15.6:0/1453946470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cb470 0x56372b7cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 -- 172.21.15.6:0/1453946470 >> 172.21.15.6:0/1453946470 conn(0x56372b7c0360 msgr2=0x56372b7d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 -- 172.21.15.6:0/1453946470 shutdown_connections 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.903+0000 7f5a819abb80 1 -- 172.21.15.6:0/1453946470 wait complete. 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 -- start start 2023-11-17T00:17:18.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 Processor -- start 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 0x56372b7d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 0x56372b837670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b7d87c0 con 0x56372b7cb470 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b837bb0 con 0x56372b7cab60 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372b837cf0 con 0x56372b7cf8b0 2023-11-17T00:17:18.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a819abb80 10 client.0 _ll_register_callbacks cb 0x7f5a50099870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 0x56372b7d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 0x56372b837670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.904+0000 7f5a677fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33314/0 (socket says 172.21.15.6:33314) 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 -- 172.21.15.6:0/302053470 learned_addr learned my addr 172.21.15.6:0/302053470 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 -- 172.21.15.6:0/302053470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 msgr2=0x56372b837670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 0x56372b837670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 -- 172.21.15.6:0/302053470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 msgr2=0x56372b7d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 0x56372b7d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 -- 172.21.15.6:0/302053470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6810d940 con 0x56372b7cab60 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a67fff700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 0x56372b837670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 0x56372b7d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a677fe700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5a58014040 tx=0x7f5a5802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ab8a83ad7abd18 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.905+0000 7f5a46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58005670 con 0x56372b7cab60 2023-11-17T00:17:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a58004390 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5a58003700 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a580038c0 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 10 client.26843 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56372b59d630 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a58004390 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a58011440 con 0x56372b7cab60 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 10 client.26843 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 10 client.26843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 20 client.26843 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b838f60 0x56372b83b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 4 client.26843 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.906+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56372b6ea660 con 0x56372b838f60 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b838f60 0x56372b83b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a5ffff700 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b838f60 0x56372b83b370 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b883b9fbbe42814d server_cookie=c48eddad6cbef51d in_seq=0 out_seq=0 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a46ffd700 10 client.26843 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a46ffd700 1 -- 172.21.15.6:0/302053470 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56372b6ea660 con 0x56372b838f60 2023-11-17T00:17:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a46ffd700 10 client.26843 handle_command_reply: tid=0 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.907+0000 7f5a819abb80 1 client.26843 shutdown 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 msgr2=0x56372b7d7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5a58014040 tx=0x7f5a5802be90 comp rx=0 tx=0).stop 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 shutdown_connections 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56372b7cf8b0 0x56372b837670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56372b7cab60 0x56372b7d7980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x56372b838f60 0x56372b83b370 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 --2- 172.21.15.6:0/302053470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x56372b7cb470 0x56372b7d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a46ffd700 0 client.26843 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 >> 172.21.15.6:0/302053470 conn(0x56372b7c0360 msgr2=0x56372b7d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 shutdown_connections 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 1 -- 172.21.15.6:0/302053470 wait complete. 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a819abb80 20 client.26843 trim_cache size 0 max 16384 2023-11-17T00:17:18.919 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:17:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 msgr2=0x7f5a540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 0x7f5a540a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a60006280 tx=0x7f5a60006210 comp rx=0 tx=0).stop 2023-11-17T00:17:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 msgr2=0x7f5a68196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.908+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e940 tx=0x7f5a5000ec50 comp rx=0 tx=0).stop 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 shutdown_connections 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f5a540a12c0 0x7f5a540a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a68105560 0x7f5a6819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a68100c50 0x7f5a68196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 --2- 172.21.15.6:0/1322733301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a681015f0 0x7f5a68196a60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 >> 172.21.15.6:0/1322733301 conn(0x7f5a680774f0 msgr2=0x7f5a68103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 shutdown_connections 2023-11-17T00:17:18.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:18.909+0000 7f5a6dfff700 1 -- 172.21.15.6:0/1322733301 wait complete. 2023-11-17T00:17:18.927 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:18 smithi006 ceph-mon[97840]: pgmap v261: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:19.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/67099940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:19.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/67099940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:19.930 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:17:19.930 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:17:19.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:19 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/67099940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.151+0000 7efdc95ff700 1 -- 172.21.15.6:0/3875966822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 msgr2=0x7efdc410f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.151+0000 7efdc95ff700 1 --2- 172.21.15.6:0/3875966822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc410f230 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00b0a0 tx=0x7efdbc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 -- 172.21.15.6:0/3875966822 shutdown_connections 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 --2- 172.21.15.6:0/3875966822 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc410f230 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 --2- 172.21.15.6:0/3875966822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 0x7efdc4072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 --2- 172.21.15.6:0/3875966822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 -- 172.21.15.6:0/3875966822 >> 172.21.15.6:0/3875966822 conn(0x7efdc4077500 msgr2=0x7efdc4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 -- 172.21.15.6:0/3875966822 shutdown_connections 2023-11-17T00:17:20.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.152+0000 7efdc95ff700 1 -- 172.21.15.6:0/3875966822 wait complete. 2023-11-17T00:17:20.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 Processor -- start 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 -- start start 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 0x7efdc4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419d650 con 0x7efdc410ce50 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419d7c0 con 0x7efdc4074b00 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc95ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419d980 con 0x7efdc40746a0 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33330/0 (socket says 172.21.15.6:33330) 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.153+0000 7efdbbfff700 1 -- 172.21.15.6:0/92149515 learned_addr learned my addr 172.21.15.6:0/92149515 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 0x7efdc4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 -- 172.21.15.6:0/92149515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 msgr2=0x7efdc4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 0x7efdc4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 -- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 msgr2=0x7efdc419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 -- 172.21.15.6:0/92149515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009d00 con 0x7efdc4074b00 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc37fe700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdbbfff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9a0 tx=0x7efdac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc0ff9700 1 -- 172.21.15.6:0/92149515 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac009790 con 0x7efdc4074b00 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc4112520 con 0x7efdc4074b00 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc0ff9700 1 -- 172.21.15.6:0/92149515 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdac010430 con 0x7efdc4074b00 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.154+0000 7efdc0ff9700 1 -- 172.21.15.6:0/92149515 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac018680 con 0x7efdc4074b00 2023-11-17T00:17:20.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.155+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc4112390 con 0x7efdc4074b00 2023-11-17T00:17:20.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.158+0000 7efdc0ff9700 1 -- 172.21.15.6:0/92149515 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7efdac0105a0 con 0x7efdc4074b00 2023-11-17T00:17:20.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 Processor -- start 2023-11-17T00:17:20.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 -- start start 2023-11-17T00:17:20.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b8930 0x5567df6b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df5b90e0 con 0x5567df6b44f0 2023-11-17T00:17:20.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df60b920 con 0x5567df6b3be0 2023-11-17T00:17:20.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efddcf58b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df604db0 con 0x5567df6b8930 2023-11-17T00:17:20.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.159+0000 7efdc0ff9700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 0x7efdb00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc0ff9700 1 -- 172.21.15.6:0/92149515 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efdac014070 con 0x7efdc4074b00 2023-11-17T00:17:20.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33340/0 (socket says 172.21.15.6:33340) 2023-11-17T00:17:20.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc37fe700 1 -- 172.21.15.6:0/287994736 learned_addr learned my addr 172.21.15.6:0/287994736 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc37fe700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 0x7efdb00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdbbfff700 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b8930 0x5567df6b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 -- 172.21.15.6:0/287994736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b8930 msgr2=0x5567df6b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b8930 0x5567df6b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 -- 172.21.15.6:0/287994736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 msgr2=0x5567df6b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.160+0000 7efdc2ffd700 1 -- 172.21.15.6:0/287994736 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567df4866b0 con 0x5567df6b44f0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6b83f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009cb0 tx=0x7efdb4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccef5792b702aa8c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efdc37fe700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 0x7efdb00a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009cd0 tx=0x7efdbc009c40 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efdb97fa700 1 -- 172.21.15.6:0/287994736 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4011d20 con 0x5567df6b44f0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efdb97fa700 1 -- 172.21.15.6:0/287994736 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb4011e80 con 0x5567df6b44f0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efdb97fa700 1 -- 172.21.15.6:0/287994736 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4005280 con 0x5567df6b44f0 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efddcf58b80 1 -- 172.21.15.6:0/287994736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 msgr2=0x5567df6b83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.161+0000 7efddcf58b80 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6b83f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009cb0 tx=0x7efdb4011440 comp rx=0 tx=0).stop 2023-11-17T00:17:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 -- 172.21.15.6:0/287994736 shutdown_connections 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b8930 0x5567df6b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 --2- 172.21.15.6:0/287994736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6b83f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 -- 172.21.15.6:0/287994736 >> 172.21.15.6:0/287994736 conn(0x5567df6a93e0 msgr2=0x5567df6aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 -- 172.21.15.6:0/287994736 shutdown_connections 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 -- 172.21.15.6:0/287994736 wait complete. 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 -- start start 2023-11-17T00:17:20.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.162+0000 7efddcf58b80 1 Processor -- start 2023-11-17T00:17:20.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b3be0 0x5567df6c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b8930 0x5567df6bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df5a6960 con 0x5567df6b44f0 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df5a6600 con 0x5567df6b8930 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df4889a0 con 0x5567df6b3be0 2023-11-17T00:17:20.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efddcf58b80 10 client.0 _ll_register_callbacks cb 0x5567df5c4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37780/0 (socket says 172.21.15.6:37780) 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efdbbfff700 1 -- 172.21.15.6:0/1518870152 learned_addr learned my addr 172.21.15.6:0/1518870152 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b3be0 0x5567df6c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.163+0000 7efdc37fe700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b8930 0x5567df6bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 -- 172.21.15.6:0/1518870152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b3be0 msgr2=0x5567df6c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b3be0 0x5567df6c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 -- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b8930 msgr2=0x5567df6bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b8930 0x5567df6bd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 -- 172.21.15.6:0/1518870152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40098e0 con 0x5567df6b44f0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdbbfff700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efdac00c2d0 tx=0x7efdac0af4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4098c5345913e5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac09a380 con 0x5567df6b44f0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efdac0a2370 con 0x5567df6b44f0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567df5c7120 con 0x5567df6b44f0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdac099410 con 0x5567df6b44f0 2023-11-17T00:17:20.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efddcf58b80 10 client.16719 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5567df5cb210 con 0x5567df6b44f0 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.164+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdac0b6080 con 0x5567df6b44f0 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdac0acb10 con 0x5567df6b44f0 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 20 client.16719 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df6cd8f0 0x5567df6cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 4 client.16719 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:20.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567df5d36e0 con 0x5567df6cd8f0 2023-11-17T00:17:20.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.165+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df6cd8f0 0x5567df6cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.166+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df6cd8f0 0x5567df6cfd00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ffc602124405e9 server_cookie=90d110a357f4d9da in_seq=0 out_seq=0 2023-11-17T00:17:20.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.166+0000 7efdaa7fc700 10 client.16719 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.166+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1518870152 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5567df5d36e0 con 0x5567df6cd8f0 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.166+0000 7efdaa7fc700 10 client.16719 handle_command_reply: tid=0 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.166+0000 7efddcf58b80 1 client.16719 shutdown 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 msgr2=0x5567df6bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efdac00c2d0 tx=0x7efdac0af4c0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 shutdown_connections 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b8930 0x5567df6bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b3be0 0x5567df6c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df6cd8f0 0x5567df6cfd00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1518870152 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b44f0 0x5567df6bd2d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 >> 172.21.15.6:0/1518870152 conn(0x5567df6a93e0 msgr2=0x5567df6b9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efdaa7fc700 0 client.16719 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 shutdown_connections 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 1 -- 172.21.15.6:0/1518870152 wait complete. 2023-11-17T00:17:20.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.167+0000 7efddcf58b80 20 client.16719 trim_cache size 0 max 16384 2023-11-17T00:17:20.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.184+0000 7efddcf58b80 1 Processor -- start 2023-11-17T00:17:20.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.184+0000 7efddcf58b80 1 -- start start 2023-11-17T00:17:20.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df6bb880 con 0x5567df6b8930 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df6b7400 con 0x5567df6b3be0 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efddcf58b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df6af790 con 0x5567df6b44f0 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37796/0 (socket says 172.21.15.6:37796) 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 -- 172.21.15.6:0/2138688815 learned_addr learned my addr 172.21.15.6:0/2138688815 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdbbfff700 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 -- 172.21.15.6:0/2138688815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 msgr2=0x5567df6aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 -- 172.21.15.6:0/2138688815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 msgr2=0x5567df6c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.185+0000 7efdc37fe700 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.186+0000 7efdc37fe700 1 -- 172.21.15.6:0/2138688815 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567df6aab80 con 0x5567df6b8930 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.186+0000 7efdc37fe700 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efdbc004e50 tx=0x7efdbc00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7524ff79cb98d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.186+0000 7efdb9ffb700 1 -- 172.21.15.6:0/2138688815 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc026070 con 0x5567df6b8930 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.186+0000 7efdb9ffb700 1 -- 172.21.15.6:0/2138688815 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdbc0078b0 con 0x5567df6b8930 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.186+0000 7efdb9ffb700 1 -- 172.21.15.6:0/2138688815 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00e040 con 0x5567df6b8930 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- 172.21.15.6:0/2138688815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 msgr2=0x5567df6bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efdbc004e50 tx=0x7efdbc00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- 172.21.15.6:0/2138688815 shutdown_connections 2023-11-17T00:17:20.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df6bfbf0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 --2- 172.21.15.6:0/2138688815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- 172.21.15.6:0/2138688815 >> 172.21.15.6:0/2138688815 conn(0x5567df6a93e0 msgr2=0x5567df6b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- 172.21.15.6:0/2138688815 shutdown_connections 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- 172.21.15.6:0/2138688815 wait complete. 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 -- start start 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.187+0000 7efddcf58b80 1 Processor -- start 2023-11-17T00:17:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df6c4210 con 0x5567df6b8930 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df721d00 con 0x5567df6b3be0 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567df721e40 con 0x5567df6b44f0 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efddcf58b80 10 client.0 _ll_register_callbacks cb 0x7efdac099880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37798/0 (socket says 172.21.15.6:37798) 2023-11-17T00:17:20.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efdc37fe700 1 -- 172.21.15.6:0/1190810400 learned_addr learned my addr 172.21.15.6:0/1190810400 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.188+0000 7efdbbfff700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 -- 172.21.15.6:0/1190810400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 msgr2=0x5567df6c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 -- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 msgr2=0x5567df6c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 -- 172.21.15.6:0/1190810400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc410fb20 con 0x5567df6b8930 2023-11-17T00:17:20.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdbbfff700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdc37fe700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efdbc007780 tx=0x7efdbc020aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da737a26bce6b4d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc026070 con 0x5567df6b8930 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567df6a6ba0 con 0x5567df6b8930 2023-11-17T00:17:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.189+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efdbc035660 con 0x5567df6b8930 2023-11-17T00:17:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdbc020ea0 con 0x5567df6b8930 2023-11-17T00:17:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 10 client.16725 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5567df432e60 con 0x5567df6b8930 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567df6a6ba0 con 0x5567df6b8930 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdbc029070 con 0x5567df6b8930 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 20 client.16725 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df723180 0x5567df725560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 4 client.16725 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5567df453ff0 con 0x5567df723180 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.190+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df723180 0x5567df725560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.191+0000 7efdaa7fc700 10 client.16725 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.191+0000 7efdc2ffd700 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df723180 0x5567df725560 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2814fb98d2d06ffe server_cookie=ad34ec7c26528eb3 in_seq=0 out_seq=0 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efdaa7fc700 1 -- 172.21.15.6:0/1190810400 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5567df453ff0 con 0x5567df723180 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efdaa7fc700 10 client.16725 handle_command_reply: tid=0 2023-11-17T00:17:20.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 client.16725 shutdown 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 msgr2=0x5567df7217c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efdbc007780 tx=0x7efdbc020aa0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 shutdown_connections 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5567df723180 0x5567df725560 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5567df6b8930 0x5567df7217c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5567df6b3be0 0x5567df6c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 --2- 172.21.15.6:0/1190810400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5567df6b44f0 0x5567df6c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efdaa7fc700 0 client.16725 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 >> 172.21.15.6:0/1190810400 conn(0x5567df6a93e0 msgr2=0x5567df6b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 shutdown_connections 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.192+0000 7efddcf58b80 1 -- 172.21.15.6:0/1190810400 wait complete. 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efddcf58b80 20 client.16725 trim_cache size 0 max 16384 2023-11-17T00:17:20.203 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:20.204 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:17:20.204 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "8af8efef-7bd6-419a-b0dd-506e2b25e5f8", 2023-11-17T00:17:20.204 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:17:20.204 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 msgr2=0x7efdb00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 0x7efdb00a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009cd0 tx=0x7efdbc009c40 comp rx=0 tx=0).stop 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 msgr2=0x7efdc4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9a0 tx=0x7efdac00ecb0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 shutdown_connections 2023-11-17T00:17:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7efdb00a12c0 0x7efdb00a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdc410ce50 0x7efdc419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdc40746a0 0x7efdc4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 --2- 172.21.15.6:0/92149515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdc4074b00 0x7efdc4198bb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 >> 172.21.15.6:0/92149515 conn(0x7efdc4077500 msgr2=0x7efdc4104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 shutdown_connections 2023-11-17T00:17:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.193+0000 7efdc95ff700 1 -- 172.21.15.6:0/92149515 wait complete. 2023-11-17T00:17:20.214 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:17:20.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 -- 172.21.15.6:0/2090233976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3504100c50 msgr2=0x7f3504101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 --2- 172.21.15.6:0/2090233976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3504100c50 0x7f3504101020 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f35000099c0 tx=0x7f3500009cd0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 -- 172.21.15.6:0/2090233976 shutdown_connections 2023-11-17T00:17:20.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 --2- 172.21.15.6:0/2090233976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3504100c50 0x7f3504101020 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 --2- 172.21.15.6:0/2090233976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 0x7f350410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 --2- 172.21.15.6:0/2090233976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35041015f0 0x7f3504104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 -- 172.21.15.6:0/2090233976 >> 172.21.15.6:0/2090233976 conn(0x7f3504077500 msgr2=0x7f3504077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 -- 172.21.15.6:0/2090233976 shutdown_connections 2023-11-17T00:17:20.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.462+0000 7f350bfff700 1 -- 172.21.15.6:0/2090233976 wait complete. 2023-11-17T00:17:20.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.463+0000 7f350bfff700 1 Processor -- start 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.463+0000 7f350bfff700 1 -- start start 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 0x7f35041963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 0x7f350419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419b3a0 con 0x7f35041015f0 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419b510 con 0x7f3504100c50 2023-11-17T00:17:20.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350419b6d0 con 0x7f3504105560 2023-11-17T00:17:20.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37812/0 (socket says 172.21.15.6:37812) 2023-11-17T00:17:20.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 -- 172.21.15.6:0/3455833982 learned_addr learned my addr 172.21.15.6:0/3455833982 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 0x7f35041963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350a59c700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 0x7f350419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 -- 172.21.15.6:0/3455833982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 msgr2=0x7f350419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 0x7f350419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 -- 172.21.15.6:0/3455833982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 msgr2=0x7f35041963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 0x7f35041963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f350959a700 1 -- 172.21.15.6:0/3455833982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000095f0 con 0x7f35041015f0 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.464+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 0x7f35041963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f350959a700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e940 tx=0x7f34f400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f34faffd700 1 -- 172.21.15.6:0/3455833982 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4009730 con 0x7f35041015f0 2023-11-17T00:17:20.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f34faffd700 1 -- 172.21.15.6:0/3455833982 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f4010430 con 0x7f35041015f0 2023-11-17T00:17:20.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504110360 con 0x7f35041015f0 2023-11-17T00:17:20.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f34faffd700 1 -- 172.21.15.6:0/3455833982 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400cea0 con 0x7f35041015f0 2023-11-17T00:17:20.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.465+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35041101d0 con 0x7f35041015f0 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.468+0000 7f34faffd700 1 -- 172.21.15.6:0/3455833982 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f34f4010810 con 0x7f35041015f0 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.469+0000 7f351f887b80 1 Processor -- start 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.469+0000 7f351f887b80 1 -- start start 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.469+0000 7f34faffd700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 0x7f34f00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f34faffd700 1 -- 172.21.15.6:0/3455833982 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f34f4014070 con 0x7f35041015f0 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74ae9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b749ea060 con 0x559b74ae9880 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74a3c8a0 con 0x559b74aedcc0 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f351f887b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74a35d30 con 0x559b74ae8f70 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f350a59c700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 0x7f34f00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f350959a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74ae9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f350959a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37818/0 (socket says 172.21.15.6:37818) 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f350959a700 1 -- 172.21.15.6:0/394320363 learned_addr learned my addr 172.21.15.6:0/394320363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.470+0000 7f350a59c700 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 -- 172.21.15.6:0/394320363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 msgr2=0x559b74aee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 -- 172.21.15.6:0/394320363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 msgr2=0x559b74aed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 -- 172.21.15.6:0/394320363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b748b7630 con 0x559b74ae8f70 2023-11-17T00:17:20.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f350959a700 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f350a59c700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 0x7f34f00a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00f540 tx=0x7f34fc009920 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f3509d9b700 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74ae9340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d7f8aae0ad62453 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f34effff700 1 -- 172.21.15.6:0/394320363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350001c050 con 0x559b74ae8f70 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f34effff700 1 -- 172.21.15.6:0/394320363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350001d040 con 0x559b74ae8f70 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.471+0000 7f34effff700 1 -- 172.21.15.6:0/394320363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500003be0 con 0x559b74ae8f70 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 -- 172.21.15.6:0/394320363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 msgr2=0x559b74ae9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74ae9340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500005fb0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 -- 172.21.15.6:0/394320363 shutdown_connections 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74ae9340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 --2- 172.21.15.6:0/394320363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae9880 0x559b74aed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 -- 172.21.15.6:0/394320363 >> 172.21.15.6:0/394320363 conn(0x559b74ada360 msgr2=0x559b74adfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 -- 172.21.15.6:0/394320363 shutdown_connections 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.472+0000 7f351f887b80 1 -- 172.21.15.6:0/394320363 wait complete. 2023-11-17T00:17:20.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 -- start start 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 Processor -- start 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74af2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b749d78e0 con 0x559b74aedcc0 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b749d7580 con 0x559b74ae9880 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b748b9920 con 0x559b74ae8f70 2023-11-17T00:17:20.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.473+0000 7f351f887b80 10 client.0 _ll_register_callbacks cb 0x559b749f5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41344/0 (socket says 172.21.15.6:41344) 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f3509d9b700 1 -- 172.21.15.6:0/3052576749 learned_addr learned my addr 172.21.15.6:0/3052576749 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350959a700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74af2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 -- 172.21.15.6:0/3052576749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 msgr2=0x559b74af5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 -- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 msgr2=0x559b74af2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74af2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 -- 172.21.15.6:0/3052576749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350410d940 con 0x559b74aedcc0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350959a700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74af2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f350a59c700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af2c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34fc019e30 tx=0x7f34fc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a444bdaca8f0cf0d server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.474+0000 7f34ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00e040 con 0x559b74aedcc0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f34fc003a00 con 0x559b74aedcc0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b74ad7b20 con 0x559b74aedcc0 2023-11-17T00:17:20.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34fc017e70 con 0x559b74aedcc0 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 10 client.16737 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559b749fc190 con 0x559b74aedcc0 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b74ad7b20 con 0x559b74aedcc0 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34fc0213f0 con 0x559b74aedcc0 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 20 client.16737 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:20.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.475+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b02cf0 0x559b74b05100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.476+0000 7f351f887b80 4 client.16737 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.476+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b749f80a0 con 0x559b74b02cf0 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.476+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b02cf0 0x559b74b05100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.476+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b02cf0 0x559b74b05100 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f0d532a0b72043 server_cookie=7322b56c8ae0b535 in_seq=0 out_seq=0 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.476+0000 7f34ed7fa700 10 client.16737 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f34ed7fa700 1 -- 172.21.15.6:0/3052576749 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559b749f80a0 con 0x559b74b02cf0 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f34ed7fa700 10 client.16737 handle_command_reply: tid=0 2023-11-17T00:17:20.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 client.16737 shutdown 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 msgr2=0x559b74af2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af2c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34fc019e30 tx=0x7f34fc004b40 comp rx=0 tx=0).stop 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 shutdown_connections 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b02cf0 0x559b74b05100 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af2c90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74af5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 --2- 172.21.15.6:0/3052576749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74af2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f34ed7fa700 0 client.16737 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.477+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 >> 172.21.15.6:0/3052576749 conn(0x559b74ada360 msgr2=0x559b74adede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.478+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 shutdown_connections 2023-11-17T00:17:20.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.478+0000 7f351f887b80 1 -- 172.21.15.6:0/3052576749 wait complete. 2023-11-17T00:17:20.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.478+0000 7f351f887b80 20 client.16737 trim_cache size 0 max 16384 2023-11-17T00:17:20.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.495+0000 7f351f887b80 1 Processor -- start 2023-11-17T00:17:20.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 -- start start 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae9880 0x559b74aeb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74ae9dd0 con 0x559b74ae8f70 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74ae11d0 con 0x559b74aedcc0 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f351f887b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74ae9fd0 con 0x559b74ae9880 2023-11-17T00:17:20.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37824/0 (socket says 172.21.15.6:37824) 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.496+0000 7f3509d9b700 1 -- 172.21.15.6:0/3000375787 learned_addr learned my addr 172.21.15.6:0/3000375787 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f350959a700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae9880 0x559b74aeb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f350a59c700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 -- 172.21.15.6:0/3000375787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae9880 msgr2=0x559b74aeb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae9880 0x559b74aeb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 -- 172.21.15.6:0/3000375787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 msgr2=0x559b74aebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 -- 172.21.15.6:0/3000375787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b74aee5b0 con 0x559b74ae8f70 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f350a59c700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f3509d9b700 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3500005b40 tx=0x7f3500005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce9f781d549d35c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f34f8ff9700 1 -- 172.21.15.6:0/3000375787 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350001c040 con 0x559b74ae8f70 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f34f8ff9700 1 -- 172.21.15.6:0/3000375787 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350000be50 con 0x559b74ae8f70 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.497+0000 7f34f8ff9700 1 -- 172.21.15.6:0/3000375787 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350000f880 con 0x559b74ae8f70 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- 172.21.15.6:0/3000375787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 msgr2=0x559b74af9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3500005b40 tx=0x7f3500005ad0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- 172.21.15.6:0/3000375787 shutdown_connections 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74ae8f70 0x559b74af9d60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74aedcc0 0x559b74aebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 --2- 172.21.15.6:0/3000375787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae9880 0x559b74aeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- 172.21.15.6:0/3000375787 >> 172.21.15.6:0/3000375787 conn(0x559b74ada360 msgr2=0x559b74aecd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- 172.21.15.6:0/3000375787 shutdown_connections 2023-11-17T00:17:20.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- 172.21.15.6:0/3000375787 wait complete. 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.498+0000 7f351f887b80 1 -- start start 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.499+0000 7f351f887b80 1 Processor -- start 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.499+0000 7f351f887b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.499+0000 7f351f887b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74b4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f351f887b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f351f887b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74af9620 con 0x559b74aedcc0 2023-11-17T00:17:20.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f351f887b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74af9790 con 0x559b74ae9880 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f351f887b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b74af9950 con 0x559b74ae8f70 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f351f887b80 10 client.0 _ll_register_callbacks cb 0x7f34fc019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:41360/0 (socket says 172.21.15.6:41360) 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 -- 172.21.15.6:0/2094860477 learned_addr learned my addr 172.21.15.6:0/2094860477 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f350a59c700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f350959a700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74b4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 -- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 msgr2=0x559b74b4f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74b4f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 -- 172.21.15.6:0/2094860477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 msgr2=0x559b74af9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 -- 172.21.15.6:0/2094860477 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b748b7630 con 0x559b74ae8f70 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f3509d9b700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3500005c00 tx=0x7f3500005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28191052ea99dd6c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.500+0000 7f34ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350001c070 con 0x559b74ae8f70 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f350001d040 con 0x559b74ae8f70 2023-11-17T00:17:20.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3500003c10 con 0x559b74ae8f70 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3500003dd0 con 0x559b74ae8f70 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f351f887b80 10 client.26851 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559b74863de0 con 0x559b74ae8f70 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f350001d040 con 0x559b74ae8f70 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.501+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f350003b030 con 0x559b74ae8f70 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 10 client.26851 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:20.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 10 client.26851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 20 client.26851 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b56cd0 0x559b74b590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 4 client.26851 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b74a04660 con 0x559b74b56cd0 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f350959a700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b56cd0 0x559b74b590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f350959a700 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b56cd0 0x559b74b590b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12a13527e103463 server_cookie=d3aa9e3d7e4e9fe2 in_seq=0 out_seq=0 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.502+0000 7f34ed7fa700 10 client.26851 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f34ed7fa700 1 -- 172.21.15.6:0/2094860477 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559b74a04660 con 0x559b74b56cd0 2023-11-17T00:17:20.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f34ed7fa700 10 client.26851 handle_command_reply: tid=0 2023-11-17T00:17:20.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 client.26851 shutdown 2023-11-17T00:17:20.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 msgr2=0x559b74b4f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3500005c00 tx=0x7f3500005c30 comp rx=0 tx=0).stop 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 shutdown_connections 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559b74b56cd0 0x559b74b590b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559b74aedcc0 0x559b74af9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f34ed7fa700 0 client.26851 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:20.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559b74ae8f70 0x559b74b4f350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 --2- 172.21.15.6:0/2094860477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559b74ae9880 0x559b74b4f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 >> 172.21.15.6:0/2094860477 conn(0x559b74ada360 msgr2=0x559b74ae7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 shutdown_connections 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 1 -- 172.21.15.6:0/2094860477 wait complete. 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.503+0000 7f351f887b80 20 client.26851 trim_cache size 0 max 16384 2023-11-17T00:17:20.516 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "8af8efef-7bd6-419a-b0dd-506e2b25e5f8": { 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "8af8efef-7bd6-419a-b0dd-506e2b25e5f8", 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:17:20.517 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:17:20.518 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:20.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 msgr2=0x7f34f00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 0x7f34f00a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00f540 tx=0x7f34fc009920 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 msgr2=0x7f3504196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e940 tx=0x7f34f400ec50 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 shutdown_connections 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f34f00a1340 0x7f34f00a37f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3504105560 0x7f350419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3504100c50 0x7f35041963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 --2- 172.21.15.6:0/3455833982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35041015f0 0x7f3504196900 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:20.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 >> 172.21.15.6:0/3455833982 conn(0x7f3504077500 msgr2=0x7f35040feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:20.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 shutdown_connections 2023-11-17T00:17:20.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:20.504+0000 7f350bfff700 1 -- 172.21.15.6:0/3455833982 wait complete. 2023-11-17T00:17:20.524 INFO:tasks.cephfs.filesystem:scrub status for tag:8af8efef-7bd6-419a-b0dd-506e2b25e5f8 - {'path': '/', 'tag': '8af8efef-7bd6-419a-b0dd-506e2b25e5f8', 'options': 'recursive,force'} 2023-11-17T00:17:20.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:20 smithi087 ceph-mon[131182]: pgmap v262: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:20.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:20 smithi186 ceph-mon[131487]: pgmap v262: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:20 smithi006 ceph-mon[97840]: pgmap v262: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:17:21.853 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:21 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:17:21.984 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:21 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:21.984 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:21 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:17:21.984 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:21 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:21.985 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:21 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:17:21.985 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:21 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:21.985 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:21 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:17:21.985 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:21 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:21.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:21 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:17:21.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:21 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:21.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:21 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:17:21.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:21 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:22 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:17:22.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:22 smithi087 ceph-mon[131182]: pgmap v263: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:22.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:22 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:17:22.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:22 smithi186 ceph-mon[131487]: pgmap v263: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:22 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:17:22.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:22 smithi006 ceph-mon[97840]: pgmap v263: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:23.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:23 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:17:23.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:23 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:17:24.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:23 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:17:24.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: pgmap v264: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:17:32.543 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:17:32.544 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:24 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.546 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:26 smithi186 ceph-mon[131487]: pgmap v265: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:17:32.557 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:28 smithi186 ceph-mon[131487]: pgmap v266: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.557 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:30 smithi186 ceph-mon[131487]: pgmap v267: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.667 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: pgmap v264: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:32.667 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:17:32.667 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:17:32.667 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:24 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:26 smithi087 ceph-mon[131182]: pgmap v265: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:28 smithi087 ceph-mon[131182]: pgmap v266: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.668 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:30 smithi087 ceph-mon[131182]: pgmap v267: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: pgmap v264: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 102 B/s wr, 2 op/s 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:17:32.669 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:24 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:17:32.670 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:26 smithi006 ceph-mon[97840]: pgmap v265: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:17:32.670 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:28 smithi006 ceph-mon[97840]: pgmap v266: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.670 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:30 smithi006 ceph-mon[97840]: pgmap v267: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:17:32.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:32 smithi006 ceph-mon[97840]: pgmap v268: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:17:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:32 smithi087 ceph-mon[131182]: pgmap v268: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:17:33.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:32 smithi186 ceph-mon[131487]: pgmap v268: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:17:34.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:34 smithi006 ceph-mon[97840]: pgmap v269: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2023-11-17T00:17:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:34 smithi087 ceph-mon[131182]: pgmap v269: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2023-11-17T00:17:35.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:34 smithi186 ceph-mon[131487]: pgmap v269: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2023-11-17T00:17:36.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:36 smithi006 ceph-mon[97840]: pgmap v270: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2023-11-17T00:17:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:36 smithi087 ceph-mon[131182]: pgmap v270: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2023-11-17T00:17:37.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:36 smithi186 ceph-mon[131487]: pgmap v270: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s 2023-11-17T00:17:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:38 smithi006 ceph-mon[97840]: pgmap v271: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:39.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:38 smithi087 ceph-mon[131182]: pgmap v271: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:39.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:38 smithi186 ceph-mon[131487]: pgmap v271: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:40.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:40 smithi006 ceph-mon[97840]: pgmap v272: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:41.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:40 smithi087 ceph-mon[131182]: pgmap v272: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:41.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:40 smithi186 ceph-mon[131487]: pgmap v272: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:42.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:42 smithi006 ceph-mon[97840]: pgmap v273: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:17:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:42 smithi087 ceph-mon[131182]: pgmap v273: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:17:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:42 smithi186 ceph-mon[131487]: pgmap v273: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:17:44.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:44 smithi006 ceph-mon[97840]: pgmap v274: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:45.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:44 smithi087 ceph-mon[131182]: pgmap v274: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:45.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:44 smithi186 ceph-mon[131487]: pgmap v274: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:17:46.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:46 smithi006 ceph-mon[97840]: pgmap v275: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:17:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:46 smithi087 ceph-mon[131182]: pgmap v275: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:17:47.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:46 smithi186 ceph-mon[131487]: pgmap v275: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-11-17T00:17:48.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:48 smithi006 ceph-mon[97840]: pgmap v276: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:48 smithi087 ceph-mon[131182]: pgmap v276: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:49.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:48 smithi186 ceph-mon[131487]: pgmap v276: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:50.525 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:17:50.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 -- 172.21.15.6:0/1853203685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f14100c50 msgr2=0x7f7f14101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 --2- 172.21.15.6:0/1853203685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f14100c50 0x7f7f14101020 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7f040099c0 tx=0x7f7f04009cd0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 -- 172.21.15.6:0/1853203685 shutdown_connections 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 --2- 172.21.15.6:0/1853203685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f14100c50 0x7f7f14101020 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 --2- 172.21.15.6:0/1853203685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 0x7f7f1410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 --2- 172.21.15.6:0/1853203685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f141015f0 0x7f7f14104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 -- 172.21.15.6:0/1853203685 >> 172.21.15.6:0/1853203685 conn(0x7f7f14077500 msgr2=0x7f7f14077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 -- 172.21.15.6:0/1853203685 shutdown_connections 2023-11-17T00:17:50.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.764+0000 7f7f195ff700 1 -- 172.21.15.6:0/1853203685 wait complete. 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.765+0000 7f7f195ff700 1 Processor -- start 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.765+0000 7f7f195ff700 1 -- start start 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f14100c50 0x7f7f14072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 0x7f7f141a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14073b60 con 0x7f7f141015f0 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a4aa0 con 0x7f7f14100c50 2023-11-17T00:17:50.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f195ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a4be0 con 0x7f7f14105560 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44254/0 (socket says 172.21.15.6:44254) 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 -- 172.21.15.6:0/2021629605 learned_addr learned my addr 172.21.15.6:0/2021629605 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f14100c50 0x7f7f14072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f137fe700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 0x7f7f141a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 -- 172.21.15.6:0/2021629605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 msgr2=0x7f7f141a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 0x7f7f141a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 -- 172.21.15.6:0/2021629605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f14100c50 msgr2=0x7f7f14072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f14100c50 0x7f7f14072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.766+0000 7f7f127fc700 1 -- 172.21.15.6:0/2021629605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040095f0 con 0x7f7f141015f0 2023-11-17T00:17:50.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7f127fc700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e940 tx=0x7f7f0800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7efbfff700 1 -- 172.21.15.6:0/2021629605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08009730 con 0x7f7f141015f0 2023-11-17T00:17:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14110310 con 0x7f7f141015f0 2023-11-17T00:17:50.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7efbfff700 1 -- 172.21.15.6:0/2021629605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0800ee90 con 0x7f7f141015f0 2023-11-17T00:17:50.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7efbfff700 1 -- 172.21.15.6:0/2021629605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08010ea0 con 0x7f7f141015f0 2023-11-17T00:17:50.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.767+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14110180 con 0x7f7f141015f0 2023-11-17T00:17:50.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.770+0000 7f7efbfff700 1 -- 172.21.15.6:0/2021629605 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7f0800cbd0 con 0x7f7f141015f0 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.771+0000 7f7f2cf08b80 1 Processor -- start 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 -- start start 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7efbfff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 0x7f7efc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7efbfff700 1 -- 172.21.15.6:0/2021629605 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f08014070 con 0x7f7f141015f0 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4653de060 con 0x55a4654d8b60 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4654308a0 con 0x55a4654dd8b0 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a465429d30 con 0x55a4654d9470 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f137fe700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 0x7f7efc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44268/0 (socket says 172.21.15.6:44268) 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f12ffd700 1 -- 172.21.15.6:0/3859563473 learned_addr learned my addr 172.21.15.6:0/3859563473 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f137fe700 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.772+0000 7f7f127fc700 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 -- 172.21.15.6:0/3859563473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 msgr2=0x55a4654dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 -- 172.21.15.6:0/3859563473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 msgr2=0x55a4654ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 -- 172.21.15.6:0/3859563473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4652ab630 con 0x55a4654d8b60 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f137fe700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 0x7f7efc0a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009550 tx=0x7f7f0c009bd0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400b580 tx=0x7f7f04005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960c1dd99303476 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7ef97fa700 1 -- 172.21.15.6:0/3859563473 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0400b990 con 0x55a4654d8b60 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7ef97fa700 1 -- 172.21.15.6:0/3859563473 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0401e040 con 0x55a4654d8b60 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.773+0000 7f7ef97fa700 1 -- 172.21.15.6:0/3859563473 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0400fd90 con 0x55a4654d8b60 2023-11-17T00:17:50.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3859563473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 msgr2=0x55a4654d8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400b580 tx=0x7f7f04005f00 comp rx=0 tx=0).stop 2023-11-17T00:17:50.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3859563473 shutdown_connections 2023-11-17T00:17:50.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d8b60 0x55a4654d8f30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3859563473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3859563473 >> 172.21.15.6:0/3859563473 conn(0x55a4654ce360 msgr2=0x55a4654d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3859563473 shutdown_connections 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.774+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3859563473 wait complete. 2023-11-17T00:17:50.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 -- start start 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 Processor -- start 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654dd8b0 0x55a4654e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4653cb8e0 con 0x55a4654d9470 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4653cb580 con 0x55a4654d8b60 2023-11-17T00:17:50.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.775+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4652ad920 con 0x55a4654dd8b0 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f2cf08b80 10 client.0 _ll_register_callbacks cb 0x55a4653e9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44278/0 (socket says 172.21.15.6:44278) 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 -- 172.21.15.6:0/1671169191 learned_addr learned my addr 172.21.15.6:0/1671169191 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f137fe700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654dd8b0 0x55a4654e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 -- 172.21.15.6:0/1671169191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654dd8b0 msgr2=0x55a4654e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654dd8b0 0x55a4654e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 -- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 msgr2=0x55a4654e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 -- 172.21.15.6:0/1671169191 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1410d940 con 0x55a4654d9470 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:50.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7f127fc700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7f08099f40 tx=0x7f7f08099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739c34c85a2b9a4a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.776+0000 7f7ef6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08099fb0 con 0x55a4654d9470 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f0809dfe0 con 0x55a4654d9470 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4654cbb20 con 0x55a4654d9470 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f080a2400 con 0x55a4654d9470 2023-11-17T00:17:50.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 10 client.16755 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a4653f0190 con 0x55a4654d9470 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f080c8020 con 0x55a4654d9470 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f08098f10 con 0x55a4654d9470 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:50.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 20 client.16755 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:50.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a4654f2870 0x55a4654f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 4 client.16755 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:50.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.777+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4653ec0a0 con 0x55a4654f2870 2023-11-17T00:17:50.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.778+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a4654f2870 0x55a4654f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.778+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a4654f2870 0x55a4654f4c80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0d839ea642d7cd server_cookie=5945421bbedad243 in_seq=0 out_seq=0 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.778+0000 7f7ef6ff5700 10 client.16755 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/1671169191 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4653ec0a0 con 0x55a4654f2870 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7ef6ff5700 10 client.16755 handle_command_reply: tid=0 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 client.16755 shutdown 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 msgr2=0x55a4654e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7f08099f40 tx=0x7f7f08099990 comp rx=0 tx=0).stop 2023-11-17T00:17:50.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 shutdown_connections 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654dd8b0 0x55a4654e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a4654f2870 0x55a4654f4c80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/1671169191 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654e2260 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7ef6ff5700 0 client.16755 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:50.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 >> 172.21.15.6:0/1671169191 conn(0x55a4654ce360 msgr2=0x55a4654ded60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 shutdown_connections 2023-11-17T00:17:50.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.779+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/1671169191 wait complete. 2023-11-17T00:17:50.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.780+0000 7f7f2cf08b80 20 client.16755 trim_cache size 0 max 16384 2023-11-17T00:17:50.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 Processor -- start 2023-11-17T00:17:50.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 -- start start 2023-11-17T00:17:50.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d8b60 0x55a4654d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4654e0800 con 0x55a4654d9470 2023-11-17T00:17:50.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4654dc3f0 con 0x55a4654dd8b0 2023-11-17T00:17:50.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.797+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4654d2a90 con 0x55a4654d8b60 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44288/0 (socket says 172.21.15.6:44288) 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 -- 172.21.15.6:0/3069813955 learned_addr learned my addr 172.21.15.6:0/3069813955 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d8b60 0x55a4654d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f137fe700 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 -- 172.21.15.6:0/3069813955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d8b60 msgr2=0x55a4654d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d8b60 0x55a4654d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 -- 172.21.15.6:0/3069813955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 msgr2=0x55a4654eea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654eea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 -- 172.21.15.6:0/3069813955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4653efee0 con 0x55a4654d9470 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7f127fc700 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f08099270 tx=0x7f7f0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a110790c7cb71584 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7ef9ffb700 1 -- 172.21.15.6:0/3069813955 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080b83c0 con 0x55a4654d9470 2023-11-17T00:17:50.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7ef9ffb700 1 -- 172.21.15.6:0/3069813955 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f080c8030 con 0x55a4654d9470 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.798+0000 7f7ef9ffb700 1 -- 172.21.15.6:0/3069813955 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080b81f0 con 0x55a4654d9470 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3069813955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 msgr2=0x55a4654d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7f08099270 tx=0x7f7f0801f000 comp rx=0 tx=0).stop 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3069813955 shutdown_connections 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654dd8b0 0x55a4654eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d8b60 0x55a4654d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/3069813955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654d9470 0x55a4654d2d30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3069813955 >> 172.21.15.6:0/3069813955 conn(0x55a4654ce360 msgr2=0x55a4654dfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3069813955 shutdown_connections 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/3069813955 wait complete. 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.799+0000 7f7f2cf08b80 1 -- start start 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 Processor -- start 2023-11-17T00:17:50.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 0x55a4654e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4654e9f50 con 0x55a4654dd8b0 2023-11-17T00:17:50.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a465545730 con 0x55a4654d8b60 2023-11-17T00:17:50.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4655458f0 con 0x55a4654d9470 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f2cf08b80 10 client.0 _ll_register_callbacks cb 0x7f7f08099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 0x55a4654e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f127fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37042/0 (socket says 172.21.15.6:37042) 2023-11-17T00:17:50.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.800+0000 7f7f127fc700 1 -- 172.21.15.6:0/393347278 learned_addr learned my addr 172.21.15.6:0/393347278 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 -- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 msgr2=0x55a4654e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 -- 172.21.15.6:0/393347278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 msgr2=0x55a4654e83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 0x55a4654e83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 -- 172.21.15.6:0/393347278 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4653f0190 con 0x55a4654d9470 2023-11-17T00:17:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f137fe700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 0x55a4654e83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f127fc700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f080964b0 tx=0x7f7f08099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b3783f066ab0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7ef6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:17:50.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08099fb0 con 0x55a4654d9470 2023-11-17T00:17:50.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f080c9080 con 0x55a4654d9470 2023-11-17T00:17:50.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.801+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f0401e040 con 0x55a4654d9470 2023-11-17T00:17:50.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f080a2400 con 0x55a4654d9470 2023-11-17T00:17:50.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 10 client.26869 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:50.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55a465240c60 con 0x55a4654d9470 2023-11-17T00:17:50.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0401e040 con 0x55a4654d9470 2023-11-17T00:17:50.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f0809af00 con 0x55a4654d9470 2023-11-17T00:17:50.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 10 client.26869 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:50.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 10 client.26869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:50.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 20 client.26869 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a465546ba0 0x55a465548f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 4 client.26869 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4653f8660 con 0x55a465546ba0 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.802+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a465546ba0 0x55a465548f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f12ffd700 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a465546ba0 0x55a465548f80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a607019493d829eb server_cookie=fb073e8e451ed452 in_seq=0 out_seq=0 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7ef6ff5700 10 client.26869 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:50.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7ef6ff5700 1 -- 172.21.15.6:0/393347278 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a4653f8660 con 0x55a465546ba0 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7ef6ff5700 10 client.26869 handle_command_reply: tid=0 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 client.26869 shutdown 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 msgr2=0x55a4654e7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f080964b0 tx=0x7f7f08099910 comp rx=0 tx=0).stop 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 shutdown_connections 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55a465546ba0 0x55a465548f80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a4654dd8b0 0x55a4654e83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7ef6ff5700 0 client.26869 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a4654d8b60 0x55a4654e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 --2- 172.21.15.6:0/393347278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4654d9470 0x55a4654e7e70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 >> 172.21.15.6:0/393347278 conn(0x55a4654ce360 msgr2=0x55a4654e4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.803+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 shutdown_connections 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f2cf08b80 1 -- 172.21.15.6:0/393347278 wait complete. 2023-11-17T00:17:50.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f2cf08b80 20 client.26869 trim_cache size 0 max 16384 2023-11-17T00:17:50.828 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:50.828 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:17:50.828 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:17:50.828 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 msgr2=0x7f7efc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 0x7f7efc0a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009550 tx=0x7f7f0c009bd0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 msgr2=0x7f7f14073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e940 tx=0x7f7f0800ec50 comp rx=0 tx=0).stop 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 shutdown_connections 2023-11-17T00:17:50.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f7efc0a1340 0x7f7efc0a37f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f14105560 0x7f7f141a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f14100c50 0x7f7f14072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 --2- 172.21.15.6:0/2021629605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f141015f0 0x7f7f14073330 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 >> 172.21.15.6:0/2021629605 conn(0x7f7f14077500 msgr2=0x7f7f140fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 shutdown_connections 2023-11-17T00:17:50.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:50.804+0000 7f7f195ff700 1 -- 172.21.15.6:0/2021629605 wait complete. 2023-11-17T00:17:50.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:17:50.832 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:17:50.895 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:50 smithi006 ceph-mon[97840]: pgmap v277: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:51.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 -- 172.21.15.6:0/1300120364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c80746a0 msgr2=0x7f30c8072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/1300120364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c80746a0 0x7f30c8072d20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009990 tx=0x7f30b8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 -- 172.21.15.6:0/1300120364 shutdown_connections 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/1300120364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c810cec0 0x7f30c810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/1300120364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c80746a0 0x7f30c8072d20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/1300120364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c8073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 -- 172.21.15.6:0/1300120364 >> 172.21.15.6:0/1300120364 conn(0x7f30c80774c0 msgr2=0x7f30c80778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 -- 172.21.15.6:0/1300120364 shutdown_connections 2023-11-17T00:17:51.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.040+0000 7f30cc9ff700 1 -- 172.21.15.6:0/1300120364 wait complete. 2023-11-17T00:17:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.041+0000 7f30cc9ff700 1 Processor -- start 2023-11-17T00:17:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.041+0000 7f30cc9ff700 1 -- start start 2023-11-17T00:17:51.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c80746a0 0x7f30c819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c810cec0 0x7f30c81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8198b50 con 0x7f30c8074b00 2023-11-17T00:17:51.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8198cc0 con 0x7f30c80746a0 2023-11-17T00:17:51.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30cc9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8198e80 con 0x7f30c810cec0 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44322/0 (socket says 172.21.15.6:44322) 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 -- 172.21.15.6:0/2048848372 learned_addr learned my addr 172.21.15.6:0/2048848372 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c659c700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c80746a0 0x7f30c819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c6d9d700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c810cec0 0x7f30c81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 -- 172.21.15.6:0/2048848372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c810cec0 msgr2=0x7f30c81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c810cec0 0x7f30c81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 -- 172.21.15.6:0/2048848372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c80746a0 msgr2=0x7f30c819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c80746a0 0x7f30c819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.042+0000 7f30c5d9b700 1 -- 172.21.15.6:0/2048848372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80095f0 con 0x7f30c8074b00 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30c5d9b700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e9a0 tx=0x7f30bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc009790 con 0x7f30c8074b00 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30bc00cbd0 con 0x7f30c8074b00 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc018680 con 0x7f30c8074b00 2023-11-17T00:17:51.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8112600 con 0x7f30c8074b00 2023-11-17T00:17:51.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.043+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8112470 con 0x7f30c8074b00 2023-11-17T00:17:51.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.044+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8112790 con 0x7f30c8074b00 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.046+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f30bc00cd40 con 0x7f30c8074b00 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.047+0000 7f30b77fe700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 0x7f30b00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.047+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f30bc014070 con 0x7f30c8074b00 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.048+0000 7f30c659c700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 0x7f30b00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.048+0000 7f30c659c700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 0x7f30b00a37f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b800b260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.051+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f30bc051730 con 0x7f30c8074b00 2023-11-17T00:17:51.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:50 smithi087 ceph-mon[131182]: pgmap v277: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:50 smithi186 ceph-mon[131487]: pgmap v277: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:51.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.313+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c8107d20 con 0x7f30c8074b00 2023-11-17T00:17:51.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.314+0000 7f30b77fe700 1 -- 172.21.15.6:0/2048848372 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7f30bc099ec0 con 0x7f30c8074b00 2023-11-17T00:17:51.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:17:51.316 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:17:51.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 msgr2=0x7f30b00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 0x7f30b00a37f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b800b260 comp rx=0 tx=0).stop 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 msgr2=0x7f30c819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e9a0 tx=0x7f30bc00ed60 comp rx=0 tx=0).stop 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 shutdown_connections 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30b00a1340 0x7f30b00a37f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30c810cec0 0x7f30c81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30c80746a0 0x7f30c819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 --2- 172.21.15.6:0/2048848372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30c8074b00 0x7f30c819e960 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 >> 172.21.15.6:0/2048848372 conn(0x7f30c80774c0 msgr2=0x7f30c8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 shutdown_connections 2023-11-17T00:17:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.316+0000 7f30cc9ff700 1 -- 172.21.15.6:0/2048848372 wait complete. 2023-11-17T00:17:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:17:51.337 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:17:51.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 -- 172.21.15.6:0/832510295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e4074b00 msgr2=0x7f35e4073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 --2- 172.21.15.6:0/832510295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e4074b00 0x7f35e4073260 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009990 tx=0x7f35d0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 -- 172.21.15.6:0/832510295 shutdown_connections 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 --2- 172.21.15.6:0/832510295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e410ce50 0x7f35e410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 --2- 172.21.15.6:0/832510295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 0x7f35e4072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 --2- 172.21.15.6:0/832510295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e4074b00 0x7f35e4073260 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 -- 172.21.15.6:0/832510295 >> 172.21.15.6:0/832510295 conn(0x7f35e40774f0 msgr2=0x7f35e40778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 -- 172.21.15.6:0/832510295 shutdown_connections 2023-11-17T00:17:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.543+0000 7f35eafff700 1 -- 172.21.15.6:0/832510295 wait complete. 2023-11-17T00:17:51.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.544+0000 7f35eafff700 1 Processor -- start 2023-11-17T00:17:51.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.544+0000 7f35eafff700 1 -- start start 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 0x7f35e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4074b00 0x7f35e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e419d590 con 0x7f35e410ce50 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e419d700 con 0x7f35e4074b00 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35eafff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e419d8c0 con 0x7f35e40746a0 2023-11-17T00:17:51.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44330/0 (socket says 172.21.15.6:44330) 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 -- 172.21.15.6:0/275101120 learned_addr learned my addr 172.21.15.6:0/275101120 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e3fff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4074b00 0x7f35e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 0x7f35e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 -- 172.21.15.6:0/275101120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 msgr2=0x7f35e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 0x7f35e41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 -- 172.21.15.6:0/275101120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4074b00 msgr2=0x7f35e4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4074b00 0x7f35e4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.545+0000 7f35e959c700 1 -- 172.21.15.6:0/275101120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00095f0 con 0x7f35e410ce50 2023-11-17T00:17:51.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35e959c700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00ef90 tx=0x7f35dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35e1ffb700 1 -- 172.21.15.6:0/275101120 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc018070 con 0x7f35e410ce50 2023-11-17T00:17:51.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35e1ffb700 1 -- 172.21.15.6:0/275101120 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35dc00f040 con 0x7f35e410ce50 2023-11-17T00:17:51.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4112530 con 0x7f35e410ce50 2023-11-17T00:17:51.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35e1ffb700 1 -- 172.21.15.6:0/275101120 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc003e00 con 0x7f35e410ce50 2023-11-17T00:17:51.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.546+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e41123a0 con 0x7f35e410ce50 2023-11-17T00:17:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.549+0000 7f35e1ffb700 1 -- 172.21.15.6:0/275101120 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f35dc00ca50 con 0x7f35e410ce50 2023-11-17T00:17:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.551+0000 7f35fe95db80 1 Processor -- start 2023-11-17T00:17:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.551+0000 7f35fe95db80 1 -- start start 2023-11-17T00:17:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e1ffb700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 0x7f35d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287bb3060 con 0x557287cae470 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287c058a0 con 0x557287cadb60 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35fe95db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287bfed30 con 0x557287cb28b0 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e1ffb700 1 -- 172.21.15.6:0/275101120 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f35dc0cccc0 con 0x7f35e410ce50 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e3fff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 0x7f35d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e959c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46704/0 (socket says 172.21.15.6:46704) 2023-11-17T00:17:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2864842901 learned_addr learned my addr 172.21.15.6:0/2864842901 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.552+0000 7f35e3fff700 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2864842901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 msgr2=0x557287cb2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2864842901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 msgr2=0x557287cb2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2864842901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557287a80630 con 0x557287cadb60 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f35d800dc70 tx=0x7f35d800df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48fe1f5f23b8d39 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35e3fff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 0x7f35d40a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e4199bc0 tx=0x7f35d000b610 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35ceffd700 1 -- 172.21.15.6:0/2864842901 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800fb70 con 0x557287cadb60 2023-11-17T00:17:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35ceffd700 1 -- 172.21.15.6:0/2864842901 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d8004cf0 con 0x557287cadb60 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.553+0000 7f35ceffd700 1 -- 172.21.15.6:0/2864842901 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8005720 con 0x557287cadb60 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 -- 172.21.15.6:0/2864842901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 msgr2=0x557287cadf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f35d800dc70 tx=0x7f35d800df80 comp rx=0 tx=0).stop 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 -- 172.21.15.6:0/2864842901 shutdown_connections 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cadf30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2864842901 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 -- 172.21.15.6:0/2864842901 >> 172.21.15.6:0/2864842901 conn(0x557287ca3360 msgr2=0x557287ca8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 -- 172.21.15.6:0/2864842901 shutdown_connections 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.554+0000 7f35fe95db80 1 -- 172.21.15.6:0/2864842901 wait complete. 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 -- start start 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 Processor -- start 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287ba08e0 con 0x557287cae470 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287ba0580 con 0x557287cadb60 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287a82920 con 0x557287cb28b0 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.555+0000 7f35fe95db80 10 client.0 _ll_register_callbacks cb 0x557287bbe780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e959c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46706/0 (socket says 172.21.15.6:46706) 2023-11-17T00:17:51.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 -- 172.21.15.6:0/1486499903 learned_addr learned my addr 172.21.15.6:0/1486499903 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e3fff700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 -- 172.21.15.6:0/1486499903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 msgr2=0x557287cb79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 -- 172.21.15.6:0/1486499903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 msgr2=0x557287cb72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 -- 172.21.15.6:0/1486499903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d800d8a0 con 0x557287cadb60 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e959c700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:51.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005530 tx=0x7f35d8005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c44a3df4bb31bd server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.556+0000 7f35c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8020070 con 0x557287cadb60 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557287ca0b20 con 0x557287cadb60 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f35d8004750 con 0x557287cadb60 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d8004910 con 0x557287cadb60 2023-11-17T00:17:51.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35fe95db80 10 client.26915 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557287bc5190 con 0x557287cadb60 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557287ca0b20 con 0x557287cadb60 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35d8003680 con 0x557287cadb60 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35fe95db80 10 client.26915 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.557+0000 7f35fe95db80 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35fe95db80 20 client.26915 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287cc7860 0x557287cc9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35fe95db80 4 client.26915 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:51.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557287bc10a0 con 0x557287cc7860 2023-11-17T00:17:51.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35e3fff700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287cc7860 0x557287cc9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35e3fff700 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287cc7860 0x557287cc9c70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8605ec33c5c8b64 server_cookie=e0a921b47d26929c in_seq=0 out_seq=0 2023-11-17T00:17:51.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.558+0000 7f35c3fff700 10 client.26915 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:51.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35c3fff700 1 -- 172.21.15.6:0/1486499903 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557287bc10a0 con 0x557287cc7860 2023-11-17T00:17:51.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35c3fff700 10 client.26915 handle_command_reply: tid=0 2023-11-17T00:17:51.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 client.26915 shutdown 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 msgr2=0x557287cba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005530 tx=0x7f35d8005560 comp rx=0 tx=0).stop 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 shutdown_connections 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cb28b0 0x557287cb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cadb60 0x557287cba090 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287cc7860 0x557287cc9c70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 --2- 172.21.15.6:0/1486499903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cb72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35c3fff700 0 client.26915 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.559+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 >> 172.21.15.6:0/1486499903 conn(0x557287ca3360 msgr2=0x557287cb3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.560+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 shutdown_connections 2023-11-17T00:17:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.560+0000 7f35fe95db80 1 -- 172.21.15.6:0/1486499903 wait complete. 2023-11-17T00:17:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.560+0000 7f35fe95db80 20 client.26915 trim_cache size 0 max 16384 2023-11-17T00:17:51.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.573+0000 7f35fe95db80 1 Processor -- start 2023-11-17T00:17:51.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.573+0000 7f35fe95db80 1 -- start start 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.573+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cae470 0x557287cb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cb28b0 0x557287cb70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35fe95db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287cb7fe0 con 0x557287cb28b0 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35fe95db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287ca2ee0 con 0x557287cae470 2023-11-17T00:17:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35fe95db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287cb7610 con 0x557287cadb60 2023-11-17T00:17:51.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e959c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cb28b0 0x557287cb70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37096/0 (socket says 172.21.15.6:37096) 2023-11-17T00:17:51.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2944573810 learned_addr learned my addr 172.21.15.6:0/2944573810 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e3fff700 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cae470 0x557287cb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2944573810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cae470 msgr2=0x557287cb6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cae470 0x557287cb6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2944573810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cb28b0 msgr2=0x557287cb70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cb28b0 0x557287cb70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 -- 172.21.15.6:0/2944573810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557287ca4b00 con 0x557287cadb60 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.574+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35d8003a60 tx=0x7f35d800df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e69d4e152894b4f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35cf7fe700 1 -- 172.21.15.6:0/2944573810 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8020070 con 0x557287cadb60 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35cf7fe700 1 -- 172.21.15.6:0/2944573810 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d8021040 con 0x557287cadb60 2023-11-17T00:17:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35cf7fe700 1 -- 172.21.15.6:0/2944573810 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800f970 con 0x557287cadb60 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 -- 172.21.15.6:0/2944573810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 msgr2=0x557287cb9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35d8003a60 tx=0x7f35d800df80 comp rx=0 tx=0).stop 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 -- 172.21.15.6:0/2944573810 shutdown_connections 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cb28b0 0x557287cb70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cb9bc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 --2- 172.21.15.6:0/2944573810 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cae470 0x557287cb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 -- 172.21.15.6:0/2944573810 >> 172.21.15.6:0/2944573810 conn(0x557287ca3360 msgr2=0x557287cb08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 -- 172.21.15.6:0/2944573810 shutdown_connections 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.575+0000 7f35fe95db80 1 -- 172.21.15.6:0/2944573810 wait complete. 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 -- start start 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 Processor -- start 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 0x557287cbced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287cbd5e0 con 0x557287cae470 2023-11-17T00:17:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287cbd750 con 0x557287cb28b0 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557287d1a6e0 con 0x557287cadb60 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.576+0000 7f35fe95db80 10 client.0 _ll_register_callbacks cb 0x7f35dc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e8d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37106/0 (socket says 172.21.15.6:37106) 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e8d9b700 1 -- 172.21.15.6:0/306809486 learned_addr learned my addr 172.21.15.6:0/306809486 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44376/0 (socket says 172.21.15.6:44376) 2023-11-17T00:17:51.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e959c700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 0x557287cbced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 -- 172.21.15.6:0/306809486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 msgr2=0x557287cbe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 -- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 msgr2=0x557287cbced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 0x557287cbced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 -- 172.21.15.6:0/306809486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557287bc5190 con 0x557287cae470 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e959c700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 0x557287cbced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35e3fff700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35d000b2e0 tx=0x7f35d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60902bfab02f9134 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.577+0000 7f35c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d002c040 con 0x557287cae470 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f35d00350d0 con 0x557287cae470 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35d8004cf0 con 0x557287cae470 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d0035290 con 0x557287cae470 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35fe95db80 10 client.16785 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557287a2cde0 con 0x557287cae470 2023-11-17T00:17:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d8004cf0 con 0x557287cae470 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.578+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35d0004d10 con 0x557287cae470 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 10 client.16785 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 20 client.16785 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287d1bc40 0x557287d1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 4 client.16785 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557287bcd660 con 0x557287d1bc40 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287d1bc40 0x557287d1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:51.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35e8d9b700 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287d1bc40 0x557287d1e050 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a861bd8ab05291f server_cookie=6f31576fe40350ad in_seq=0 out_seq=0 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.579+0000 7f35c3fff700 10 client.16785 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35c3fff700 1 -- 172.21.15.6:0/306809486 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557287bcd660 con 0x557287d1bc40 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35c3fff700 10 client.16785 handle_command_reply: tid=0 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 client.16785 shutdown 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 msgr2=0x557287cbc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35d000b2e0 tx=0x7f35d0005fd0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 shutdown_connections 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x557287cb28b0 0x557287cbced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557287cadb60 0x557287cbe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x557287d1bc40 0x557287d1e050 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 --2- 172.21.15.6:0/306809486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557287cae470 0x557287cbc990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35c3fff700 0 client.16785 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 >> 172.21.15.6:0/306809486 conn(0x557287ca3360 msgr2=0x557287cac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 shutdown_connections 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 1 -- 172.21.15.6:0/306809486 wait complete. 2023-11-17T00:17:51.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.580+0000 7f35fe95db80 20 client.16785 trim_cache size 0 max 16384 2023-11-17T00:17:51.592 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 msgr2=0x7f35d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 0x7f35d40a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e4199bc0 tx=0x7f35d000b610 comp rx=0 tx=0).stop 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 msgr2=0x7f35e419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00ef90 tx=0x7f35dc00c3a0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 shutdown_connections 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f35d40a1340 0x7f35d40a37f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e410ce50 0x7f35e419cf50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35e40746a0 0x7f35e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 --2- 172.21.15.6:0/275101120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e4074b00 0x7f35e4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:51.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 >> 172.21.15.6:0/275101120 conn(0x7f35e40774f0 msgr2=0x7f35e4104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:51.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 shutdown_connections 2023-11-17T00:17:51.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:51.581+0000 7f35eafff700 1 -- 172.21.15.6:0/275101120 wait complete. 2023-11-17T00:17:51.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:51 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2048848372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:52.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:51 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2048848372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:52.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:51 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2048848372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:17:52.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:17:52.604 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:17:52.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 -- 172.21.15.6:0/3021982977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8218105560 msgr2=0x7f821810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 --2- 172.21.15.6:0/3021982977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8218105560 0x7f821810d130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f821000b0d0 tx=0x7f821000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 -- 172.21.15.6:0/3021982977 shutdown_connections 2023-11-17T00:17:52.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 --2- 172.21.15.6:0/3021982977 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8218105560 0x7f821810d130 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 --2- 172.21.15.6:0/3021982977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218100c50 0x7f8218101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 --2- 172.21.15.6:0/3021982977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82181015f0 0x7f8218104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 -- 172.21.15.6:0/3021982977 >> 172.21.15.6:0/3021982977 conn(0x7f8218077500 msgr2=0x7f8218077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 -- 172.21.15.6:0/3021982977 shutdown_connections 2023-11-17T00:17:52.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.826+0000 7f821f9ff700 1 -- 172.21.15.6:0/3021982977 wait complete. 2023-11-17T00:17:52.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 Processor -- start 2023-11-17T00:17:52.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 -- start start 2023-11-17T00:17:52.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 0x7f8218196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 0x7f8218196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 0x7f821819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821819b530 con 0x7f82181015f0 2023-11-17T00:17:52.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821819b6a0 con 0x7f8218100c50 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.827+0000 7f821f9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821819b860 con 0x7f8218105560 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821cf9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 0x7f8218196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 0x7f8218196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821cf9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 0x7f8218196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44394/0 (socket says 172.21.15.6:44394) 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821cf9a700 1 -- 172.21.15.6:0/2107904241 learned_addr learned my addr 172.21.15.6:0/2107904241 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821df9c700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 0x7f821819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 -- 172.21.15.6:0/2107904241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 msgr2=0x7f821819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 0x7f821819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 -- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 msgr2=0x7f8218196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 0x7f8218196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821d79b700 1 -- 172.21.15.6:0/2107904241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210009d00 con 0x7f8218100c50 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.828+0000 7f821df9c700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 0x7f821819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f821d79b700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 0x7f8218196550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9f0 tx=0x7f821400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f820e7fc700 1 -- 172.21.15.6:0/2107904241 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400c9c0 con 0x7f8218100c50 2023-11-17T00:17:52.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8218110360 con 0x7f8218100c50 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f820e7fc700 1 -- 172.21.15.6:0/2107904241 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8214010430 con 0x7f8218100c50 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f820e7fc700 1 -- 172.21.15.6:0/2107904241 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214018680 con 0x7f8218100c50 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.829+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82181101d0 con 0x7f8218100c50 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.832+0000 7f820e7fc700 1 -- 172.21.15.6:0/2107904241 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8214010860 con 0x7f8218100c50 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.833+0000 7f82333f5b80 1 Processor -- start 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.833+0000 7f82333f5b80 1 -- start start 2023-11-17T00:17:52.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.833+0000 7f820e7fc700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 0x7f82040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f820e7fc700 1 -- 172.21.15.6:0/2107904241 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8214014070 con 0x7f8218100c50 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6d98800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6d99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6c950e0 con 0x55bfb6d94900 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6ce7920 con 0x55bfb6d93ff0 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f82333f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6ce0db0 con 0x55bfb6d98d40 2023-11-17T00:17:52.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821df9c700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 0x7f82040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46744/0 (socket says 172.21.15.6:46744) 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 -- 172.21.15.6:0/3407417178 learned_addr learned my addr 172.21.15.6:0/3407417178 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821cf9a700 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6d98800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 -- 172.21.15.6:0/3407417178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 msgr2=0x55bfb6d99190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6d99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 -- 172.21.15.6:0/3407417178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 msgr2=0x55bfb6d98800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6d98800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.834+0000 7f821d79b700 1 -- 172.21.15.6:0/3407417178 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb6b626b0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f821d79b700 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8214096960 tx=0x7f82140998d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7325819e7a2d51ac server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f821df9c700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 0x7f82040a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8210009040 tx=0x7f8210009340 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f82037fe700 1 -- 172.21.15.6:0/3407417178 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140a2340 con 0x55bfb6d93ff0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f82037fe700 1 -- 172.21.15.6:0/3407417178 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82140105a0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f82037fe700 1 -- 172.21.15.6:0/3407417178 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140a1eb0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f82333f5b80 1 -- 172.21.15.6:0/3407417178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 msgr2=0x55bfb6d943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.835+0000 7f82333f5b80 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8214096960 tx=0x7f82140998d0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 -- 172.21.15.6:0/3407417178 shutdown_connections 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6d99190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6d943c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 --2- 172.21.15.6:0/3407417178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6d98800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 -- 172.21.15.6:0/3407417178 >> 172.21.15.6:0/3407417178 conn(0x55bfb6d853e0 msgr2=0x55bfb6d8abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 -- 172.21.15.6:0/3407417178 shutdown_connections 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 -- 172.21.15.6:0/3407417178 wait complete. 2023-11-17T00:17:52.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.836+0000 7f82333f5b80 1 -- start start 2023-11-17T00:17:52.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 Processor -- start 2023-11-17T00:17:52.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 0x55bfb6d9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6c82960 con 0x55bfb6d93ff0 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6c82600 con 0x55bfb6d98d40 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6b649a0 con 0x55bfb6d94900 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f82333f5b80 10 client.0 _ll_register_callbacks cb 0x55bfb6ca0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f821d79b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f821d79b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44406/0 (socket says 172.21.15.6:44406) 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f821d79b700 1 -- 172.21.15.6:0/170784365 learned_addr learned my addr 172.21.15.6:0/170784365 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:52.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.837+0000 7f821df9c700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821cf9a700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 0x55bfb6d9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 -- 172.21.15.6:0/170784365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 msgr2=0x55bfb6d9d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 0x55bfb6d9d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 -- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 msgr2=0x55bfb6d9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 -- 172.21.15.6:0/170784365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82181101d0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821cf9a700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 0x55bfb6d9d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:52.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f821d79b700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8214011fd0 tx=0x7f8214099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4acbeafff85e4fb server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f8200ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140a2340 con 0x55bfb6d93ff0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f82140a20c0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfb6d82ba0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f821400cb20 con 0x55bfb6d93ff0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f82333f5b80 10 client.16797 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.838+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55bfb6ca7210 con 0x55bfb6d93ff0 2023-11-17T00:17:52.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfb6d82ba0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8214095e80 con 0x55bfb6d93ff0 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 20 client.16797 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6dad9a0 0x55bfb6dafd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 4 client.16797 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb6ca3120 con 0x55bfb6dad9a0 2023-11-17T00:17:52.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.839+0000 7f821cf9a700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6dad9a0 0x55bfb6dafd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.840+0000 7f821cf9a700 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6dad9a0 0x55bfb6dafd80 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21bd2e218f85565 server_cookie=ee721e4f52833834 in_seq=0 out_seq=0 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.840+0000 7f8200ff9700 10 client.16797 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f8200ff9700 1 -- 172.21.15.6:0/170784365 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bfb6ca3120 con 0x55bfb6dad9a0 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f8200ff9700 10 client.16797 handle_command_reply: tid=0 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 client.16797 shutdown 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 msgr2=0x55bfb6da04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8214011fd0 tx=0x7f8214099610 comp rx=0 tx=0).stop 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 shutdown_connections 2023-11-17T00:17:52.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6dad9a0 0x55bfb6dafd80 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da04d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f8200ff9700 0 client.16797 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 --2- 172.21.15.6:0/170784365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d94900 0x55bfb6d9d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 >> 172.21.15.6:0/170784365 conn(0x55bfb6d853e0 msgr2=0x55bfb6d9a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 shutdown_connections 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 1 -- 172.21.15.6:0/170784365 wait complete. 2023-11-17T00:17:52.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.841+0000 7f82333f5b80 20 client.16797 trim_cache size 0 max 16384 2023-11-17T00:17:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.859+0000 7f82333f5b80 1 Processor -- start 2023-11-17T00:17:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.859+0000 7f82333f5b80 1 -- start start 2023-11-17T00:17:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.859+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.859+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f82333f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6d84f60 con 0x55bfb6d94900 2023-11-17T00:17:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f82333f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6d8b9c0 con 0x55bfb6d98d40 2023-11-17T00:17:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f82333f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6d9bcb0 con 0x55bfb6d93ff0 2023-11-17T00:17:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44412/0 (socket says 172.21.15.6:44412) 2023-11-17T00:17:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 -- 172.21.15.6:0/1257665080 learned_addr learned my addr 172.21.15.6:0/1257665080 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821d79b700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821df9c700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 -- 172.21.15.6:0/1257665080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d93ff0 msgr2=0x55bfb6da4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 -- 172.21.15.6:0/1257665080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 msgr2=0x55bfb6d9b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 -- 172.21.15.6:0/1257665080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb6d86b80 con 0x55bfb6d94900 2023-11-17T00:17:52.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821df9c700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:52.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.860+0000 7f821cf9a700 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8208005d90 tx=0x7f8208009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4519c4e829902f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f8203fff700 1 -- 172.21.15.6:0/1257665080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820801e070 con 0x55bfb6d94900 2023-11-17T00:17:52.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f8203fff700 1 -- 172.21.15.6:0/1257665080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8208017040 con 0x55bfb6d94900 2023-11-17T00:17:52.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f8203fff700 1 -- 172.21.15.6:0/1257665080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208010430 con 0x55bfb6d94900 2023-11-17T00:17:52.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 -- 172.21.15.6:0/1257665080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 msgr2=0x55bfb6da0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8208005d90 tx=0x7f8208009260 comp rx=0 tx=0).stop 2023-11-17T00:17:52.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 -- 172.21.15.6:0/1257665080 shutdown_connections 2023-11-17T00:17:52.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d98d40 0x55bfb6d9b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 --2- 172.21.15.6:0/1257665080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6d94900 0x55bfb6da0070 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 -- 172.21.15.6:0/1257665080 >> 172.21.15.6:0/1257665080 conn(0x55bfb6d853e0 msgr2=0x55bfb6d96dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 -- 172.21.15.6:0/1257665080 shutdown_connections 2023-11-17T00:17:52.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.861+0000 7f82333f5b80 1 -- 172.21.15.6:0/1257665080 wait complete. 2023-11-17T00:17:52.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 -- start start 2023-11-17T00:17:52.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 Processor -- start 2023-11-17T00:17:52.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6da3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6da5450 con 0x55bfb6da3f70 2023-11-17T00:17:52.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.862+0000 7f82333f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6e01d80 con 0x55bfb6d93ff0 2023-11-17T00:17:52.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f82333f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb6e01f10 con 0x55bfb6d98d40 2023-11-17T00:17:52.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f82333f5b80 10 client.0 _ll_register_callbacks cb 0x55bfb6ca0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:52.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44422/0 (socket says 172.21.15.6:44422) 2023-11-17T00:17:52.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 -- 172.21.15.6:0/876963030 learned_addr learned my addr 172.21.15.6:0/876963030 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:52.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821d79b700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821cf9a700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6da3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 -- 172.21.15.6:0/876963030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 msgr2=0x55bfb6da3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6da3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 -- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 msgr2=0x55bfb6da34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 -- 172.21.15.6:0/876963030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8208009c70 con 0x55bfb6da3f70 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821d79b700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f821df9c700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8210015000 tx=0x7f821000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad278b024992f361 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.863+0000 7f8200ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:17:52.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821000e040 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82140105a0 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f82100050b0 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8210007970 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f82333f5b80 10 client.16803 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55bfb6b0ee60 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.864+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82140105a0 con 0x55bfb6da3f70 2023-11-17T00:17:52.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8210030020 con 0x55bfb6da3f70 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 20 client.16803 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6e036a0 0x55bfb6e05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 4 client.16803 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bfb6caf6e0 con 0x55bfb6e036a0 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f821d79b700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6e036a0 0x55bfb6e05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:52.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.866+0000 7f8200ff9700 10 client.16803 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.865+0000 7f821d79b700 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6e036a0 0x55bfb6e05ab0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6b4ddb53df9aa5 server_cookie=8e0b91ee7b0797b0 in_seq=0 out_seq=0 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f8200ff9700 1 -- 172.21.15.6:0/876963030 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfb6caf6e0 con 0x55bfb6e036a0 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f8200ff9700 10 client.16803 handle_command_reply: tid=0 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 client.16803 shutdown 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 msgr2=0x55bfb6e01840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8210015000 tx=0x7f821000b3e0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 shutdown_connections 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55bfb6e036a0 0x55bfb6e05ab0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bfb6da3f70 0x55bfb6e01840 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55bfb6d93ff0 0x55bfb6da34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 --2- 172.21.15.6:0/876963030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bfb6d98d40 0x55bfb6da3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 >> 172.21.15.6:0/876963030 conn(0x55bfb6d853e0 msgr2=0x55bfb6d8a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f8200ff9700 0 client.16803 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 shutdown_connections 2023-11-17T00:17:52.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.867+0000 7f82333f5b80 1 -- 172.21.15.6:0/876963030 wait complete. 2023-11-17T00:17:52.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f82333f5b80 20 client.16803 trim_cache size 0 max 16384 2023-11-17T00:17:52.898 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:52.898 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:17:52.898 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "d5748eb6-08ba-4a1b-9253-9a142bae2c75", 2023-11-17T00:17:52.898 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:17:52.899 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 msgr2=0x7f82040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 0x7f82040a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8210009040 tx=0x7f8210009340 comp rx=0 tx=0).stop 2023-11-17T00:17:52.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 msgr2=0x7f8218196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 0x7f8218196550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9f0 tx=0x7f821400ed00 comp rx=0 tx=0).stop 2023-11-17T00:17:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 shutdown_connections 2023-11-17T00:17:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f82040a12c0 0x7f82040a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8218105560 0x7f821819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8218100c50 0x7f8218196550 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 --2- 172.21.15.6:0/2107904241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82181015f0 0x7f8218196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 >> 172.21.15.6:0/2107904241 conn(0x7f8218077500 msgr2=0x7f8218103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 shutdown_connections 2023-11-17T00:17:52.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:52.868+0000 7f821f9ff700 1 -- 172.21.15.6:0/2107904241 wait complete. 2023-11-17T00:17:52.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:17:52.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:52 smithi006 ceph-mon[97840]: pgmap v278: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:53.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:52 smithi087 ceph-mon[131182]: pgmap v278: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.110+0000 7ffa3bfff700 1 -- 172.21.15.6:0/3471801330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 msgr2=0x7ffa3410f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.110+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/3471801330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3410f230 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00b0a0 tx=0x7ffa2c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 -- 172.21.15.6:0/3471801330 shutdown_connections 2023-11-17T00:17:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/3471801330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3410f230 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/3471801330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa340746a0 0x7ffa34072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/3471801330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa34074b00 0x7ffa34073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 -- 172.21.15.6:0/3471801330 >> 172.21.15.6:0/3471801330 conn(0x7ffa34077500 msgr2=0x7ffa34077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 -- 172.21.15.6:0/3471801330 shutdown_connections 2023-11-17T00:17:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.111+0000 7ffa3bfff700 1 -- 172.21.15.6:0/3471801330 wait complete. 2023-11-17T00:17:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.112+0000 7ffa3bfff700 1 Processor -- start 2023-11-17T00:17:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.112+0000 7ffa3bfff700 1 -- start start 2023-11-17T00:17:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa340746a0 0x7ffa34198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa34074b00 0x7ffa34198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3419d650 con 0x7ffa3410ce50 2023-11-17T00:17:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3419d7c0 con 0x7ffa340746a0 2023-11-17T00:17:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3419d980 con 0x7ffa34074b00 2023-11-17T00:17:53.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:44428/0 (socket says 172.21.15.6:44428) 2023-11-17T00:17:53.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3a59c700 1 -- 172.21.15.6:0/2759471213 learned_addr learned my addr 172.21.15.6:0/2759471213 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:53.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa340746a0 0x7ffa34198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.113+0000 7ffa3959a700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa34074b00 0x7ffa34198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 -- 172.21.15.6:0/2759471213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa34074b00 msgr2=0x7ffa34198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa34074b00 0x7ffa34198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 -- 172.21.15.6:0/2759471213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa340746a0 msgr2=0x7ffa34198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa340746a0 0x7ffa34198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 -- 172.21.15.6:0/2759471213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c009d00 con 0x7ffa3410ce50 2023-11-17T00:17:53.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009fd0 tx=0x7ffa2c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa2affd700 1 -- 172.21.15.6:0/2759471213 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00e040 con 0x7ffa3410ce50 2023-11-17T00:17:53.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa2affd700 1 -- 172.21.15.6:0/2759471213 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa2c0094a0 con 0x7ffa3410ce50 2023-11-17T00:17:53.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa34112520 con 0x7ffa3410ce50 2023-11-17T00:17:53.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa2affd700 1 -- 172.21.15.6:0/2759471213 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c01b6c0 con 0x7ffa3410ce50 2023-11-17T00:17:53.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.114+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa34112390 con 0x7ffa3410ce50 2023-11-17T00:17:53.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.117+0000 7ffa2affd700 1 -- 172.21.15.6:0/2759471213 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ffa2c01d470 con 0x7ffa3410ce50 2023-11-17T00:17:53.135 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:52 smithi186 ceph-mon[131487]: pgmap v278: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:53.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.119+0000 7ffa4f8f5b80 1 Processor -- start 2023-11-17T00:17:53.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 -- start start 2023-11-17T00:17:53.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa2affd700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 0x7ffa200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d71af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d71f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d71fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa2affd700 1 -- 172.21.15.6:0/2759471213 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffa2c0ce330 con 0x7ffa3410ce50 2023-11-17T00:17:53.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d620060 con 0x55c99d71f8b0 2023-11-17T00:17:53.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d6728a0 con 0x55c99d71b470 2023-11-17T00:17:53.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d66bd30 con 0x55c99d71ab60 2023-11-17T00:17:53.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.120+0000 7ffa3959a700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 0x7ffa200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3959a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d71f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d71fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d71af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3959a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d71f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46814/0 (socket says 172.21.15.6:46814) 2023-11-17T00:17:53.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d71af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37174/0 (socket says 172.21.15.6:37174) 2023-11-17T00:17:53.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3959a700 1 -- 172.21.15.6:0/3424480248 learned_addr learned my addr 172.21.15.6:0/3424480248 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:53.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 -- 172.21.15.6:0/3424480248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 msgr2=0x55c99d71af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d71af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 -- 172.21.15.6:0/3424480248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 msgr2=0x55c99d71f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d71f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 -- 172.21.15.6:0/3424480248 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c99d4ed630 con 0x55c99d71f8b0 2023-11-17T00:17:53.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d71fd00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cb680 tx=0x7ffa2c09abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d2b86d834bacf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa3959a700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 0x7ffa200a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa24012960 tx=0x7ffa24009b60 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa1ffff700 1 -- 172.21.15.6:0/3424480248 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c01a3c0 con 0x55c99d71f8b0 2023-11-17T00:17:53.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa1ffff700 1 -- 172.21.15.6:0/3424480248 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa2c09b090 con 0x55c99d71f8b0 2023-11-17T00:17:53.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.121+0000 7ffa1ffff700 1 -- 172.21.15.6:0/3424480248 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c09b2f0 con 0x55c99d71f8b0 2023-11-17T00:17:53.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3424480248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 msgr2=0x55c99d71fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d71fd00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cb680 tx=0x7ffa2c09abd0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3424480248 shutdown_connections 2023-11-17T00:17:53.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d71fd00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d71af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3424480248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d71f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3424480248 >> 172.21.15.6:0/3424480248 conn(0x55c99d710360 msgr2=0x55c99d715b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3424480248 shutdown_connections 2023-11-17T00:17:53.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.122+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3424480248 wait complete. 2023-11-17T00:17:53.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.123+0000 7ffa4f8f5b80 1 -- start start 2023-11-17T00:17:53.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.123+0000 7ffa4f8f5b80 1 Processor -- start 2023-11-17T00:17:53.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d727090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d724ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d60d8e0 con 0x55c99d71f8b0 2023-11-17T00:17:53.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d60d580 con 0x55c99d71b470 2023-11-17T00:17:53.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d4ef920 con 0x55c99d71ab60 2023-11-17T00:17:53.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa4f8f5b80 10 client.0 _ll_register_callbacks cb 0x55c99d62b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:53.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46830/0 (socket says 172.21.15.6:46830) 2023-11-17T00:17:53.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 -- 172.21.15.6:0/634751133 learned_addr learned my addr 172.21.15.6:0/634751133 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d727090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d724ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 -- 172.21.15.6:0/634751133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 msgr2=0x55c99d727090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d727090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 -- 172.21.15.6:0/634751133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 msgr2=0x55c99d724ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d724ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.124+0000 7ffa3959a700 1 -- 172.21.15.6:0/634751133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa34112390 con 0x55c99d71b470 2023-11-17T00:17:53.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa3959a700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa24012390 tx=0x7ffa240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2360071cdd664b52 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:17:53.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24010640 con 0x55c99d71b470 2023-11-17T00:17:53.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c99d70db20 con 0x55c99d71b470 2023-11-17T00:17:53.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffa24013e40 con 0x55c99d71b470 2023-11-17T00:17:53.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa240107a0 con 0x55c99d71b470 2023-11-17T00:17:53.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa4f8f5b80 10 client.26951 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:53.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.125+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c99d632190 con 0x55c99d71b470 2023-11-17T00:17:53.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c99d70db20 con 0x55c99d71b470 2023-11-17T00:17:53.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa2401e7f0 con 0x55c99d71b470 2023-11-17T00:17:53.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 10 client.26951 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 20 client.26951 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d7346d0 0x55c99d736ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 4 client.26951 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c99d62e0a0 con 0x55c99d7346d0 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.126+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d7346d0 0x55c99d736ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.127+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d7346d0 0x55c99d736ae0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d65d0851922f6e6 server_cookie=2d2fcff2073ba39a in_seq=0 out_seq=0 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.127+0000 7ffa1d7fa700 10 client.26951 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:53.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/634751133 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c99d62e0a0 con 0x55c99d7346d0 2023-11-17T00:17:53.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa1d7fa700 10 client.26951 handle_command_reply: tid=0 2023-11-17T00:17:53.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 client.26951 shutdown 2023-11-17T00:17:53.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 msgr2=0x55c99d724380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa24012390 tx=0x7ffa240047c0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 shutdown_connections 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d7346d0 0x55c99d736ae0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d724ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71ab60 0x55c99d727090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa1d7fa700 0 client.26951 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/634751133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71b470 0x55c99d724380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 >> 172.21.15.6:0/634751133 conn(0x55c99d710360 msgr2=0x55c99d720d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 shutdown_connections 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/634751133 wait complete. 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.128+0000 7ffa4f8f5b80 20 client.26951 trim_cache size 0 max 16384 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.146+0000 7ffa4f8f5b80 1 Processor -- start 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 -- start start 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 0x55c99d72b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71f8b0 0x55c99d726c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d717480 con 0x55c99d71b470 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d722900 con 0x55c99d71ab60 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d71e4e0 con 0x55c99d71f8b0 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46838/0 (socket says 172.21.15.6:46838) 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa39d9b700 1 -- 172.21.15.6:0/1810938042 learned_addr learned my addr 172.21.15.6:0/1810938042 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71f8b0 0x55c99d726c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.147+0000 7ffa3959a700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 0x55c99d72b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 -- 172.21.15.6:0/1810938042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71f8b0 msgr2=0x55c99d726c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71f8b0 0x55c99d726c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 -- 172.21.15.6:0/1810938042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 msgr2=0x55c99d72b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 0x55c99d72b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 -- 172.21.15.6:0/1810938042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c99d711b00 con 0x55c99d71ab60 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa3959a700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 0x55c99d72b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa30000c00 tx=0x7ffa30009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb0dd510c612df7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa28ff9700 1 -- 172.21.15.6:0/1810938042 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30017070 con 0x55c99d71ab60 2023-11-17T00:17:53.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa28ff9700 1 -- 172.21.15.6:0/1810938042 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa30018040 con 0x55c99d71ab60 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.148+0000 7ffa28ff9700 1 -- 172.21.15.6:0/1810938042 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30010720 con 0x55c99d71ab60 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/1810938042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 msgr2=0x55c99d715be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa30000c00 tx=0x7ffa30009260 comp rx=0 tx=0).stop 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/1810938042 shutdown_connections 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71f8b0 0x55c99d726c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d715be0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/1810938042 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71b470 0x55c99d72b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/1810938042 >> 172.21.15.6:0/1810938042 conn(0x55c99d710360 msgr2=0x55c99d7309f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/1810938042 shutdown_connections 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.149+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/1810938042 wait complete. 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 -- start start 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 Processor -- start 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d7240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 0x55c99d722c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d7231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d723780 con 0x55c99d71f8b0 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d7238f0 con 0x55c99d71ab60 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c99d723ab0 con 0x55c99d71b470 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa4f8f5b80 10 client.0 _ll_register_callbacks cb 0x7ffa240096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d7240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa3959a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 0x55c99d722c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.150+0000 7ffa3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d7231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa39d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d7240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:46852/0 (socket says 172.21.15.6:46852) 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa39d9b700 1 -- 172.21.15.6:0/3079140764 learned_addr learned my addr 172.21.15.6:0/3079140764 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:17:53.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 -- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 msgr2=0x55c99d7240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d7240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 -- 172.21.15.6:0/3079140764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 msgr2=0x55c99d7231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d7231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 -- 172.21.15.6:0/3079140764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa30004040 con 0x55c99d71b470 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3a59c700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d7231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa3959a700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 0x55c99d722c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa2401ebb0 tx=0x7ffa2400e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9584948d10597cab server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:17:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2401ee70 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa2c09b090 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.151+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffa24012660 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa2401e420 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 10 client.26926 fetch_fsmap learned FSMap version 22 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c99d499de0 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa2c09b090 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa24010bb0 con 0x55c99d71b470 2023-11-17T00:17:53.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 10 client.26926 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:17:53.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 10 client.26926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:17:53.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 20 client.26926 populate_metadata read hostname 'smithi006' 2023-11-17T00:17:53.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d789a20 0x55c99d78be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:17:53.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 4 client.26926 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.152+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c99d63a660 con 0x55c99d789a20 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d789a20 0x55c99d78be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa39d9b700 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d789a20 0x55c99d78be00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdaac7eb37b9a68 server_cookie=d6183c4ac9df3ac7 in_seq=0 out_seq=0 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa1d7fa700 10 client.26926 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa1d7fa700 1 -- 172.21.15.6:0/3079140764 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c99d63a660 con 0x55c99d789a20 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa1d7fa700 10 client.26926 handle_command_reply: tid=0 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.153+0000 7ffa4f8f5b80 1 client.26926 shutdown 2023-11-17T00:17:53.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 msgr2=0x55c99d722c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 0x55c99d722c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa2401ebb0 tx=0x7ffa2400e630 comp rx=0 tx=0).stop 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 shutdown_connections 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x55c99d789a20 0x55c99d78be00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c99d71f8b0 0x55c99d7231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55c99d71ab60 0x55c99d7240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 --2- 172.21.15.6:0/3079140764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c99d71b470 0x55c99d722c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa1d7fa700 0 client.26926 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:17:53.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 >> 172.21.15.6:0/3079140764 conn(0x55c99d710360 msgr2=0x55c99d71e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 shutdown_connections 2023-11-17T00:17:53.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 1 -- 172.21.15.6:0/3079140764 wait complete. 2023-11-17T00:17:53.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa4f8f5b80 20 client.26926 trim_cache size 0 max 16384 2023-11-17T00:17:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:17:53.238 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: "d5748eb6-08ba-4a1b-9253-9a142bae2c75": { 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "d5748eb6-08ba-4a1b-9253-9a142bae2c75", 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:17:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 msgr2=0x7ffa200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 0x7ffa200a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa24012960 tx=0x7ffa24009b60 comp rx=0 tx=0).stop 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.154+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 msgr2=0x7ffa3419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009fd0 tx=0x7ffa2c00bce0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 shutdown_connections 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7ffa200a1340 0x7ffa200a37f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa3410ce50 0x7ffa3419d010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa340746a0 0x7ffa34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 --2- 172.21.15.6:0/2759471213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa34074b00 0x7ffa34198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:17:53.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 >> 172.21.15.6:0/2759471213 conn(0x7ffa34077500 msgr2=0x7ffa34104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:17:53.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 shutdown_connections 2023-11-17T00:17:53.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:17:53.155+0000 7ffa3bfff700 1 -- 172.21.15.6:0/2759471213 wait complete. 2023-11-17T00:17:53.242 INFO:tasks.cephfs.filesystem:scrub status for tag:d5748eb6-08ba-4a1b-9253-9a142bae2c75 - {'path': '/', 'tag': 'd5748eb6-08ba-4a1b-9253-9a142bae2c75', 'options': 'recursive,force'} 2023-11-17T00:17:54.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:54 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:17:54.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:54 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:54 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:17:54.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:54 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:54 smithi087 ceph-mon[131182]: pgmap v279: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:54.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:54 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:17:54.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:54 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:54 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:17:54.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:54 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:54 smithi186 ceph-mon[131487]: pgmap v279: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:54.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:54 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:17:54.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:54 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:54 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:17:54.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:54 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:17:54.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:54 smithi006 ceph-mon[97840]: pgmap v279: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2023-11-17T00:17:56.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:56 smithi087 ceph-mon[131182]: pgmap v280: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2023-11-17T00:17:56.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:56 smithi186 ceph-mon[131487]: pgmap v280: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2023-11-17T00:17:56.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:56 smithi006 ceph-mon[97840]: pgmap v280: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 3 op/s 2023-11-17T00:17:57.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:57 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:17:57.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:57 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:17:57.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:57 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:17:58.851 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:17:58 smithi087 ceph-mon[131182]: pgmap v281: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:17:58.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:17:58 smithi186 ceph-mon[131487]: pgmap v281: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:17:58.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:17:58 smithi006 ceph-mon[97840]: pgmap v281: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 85 B/s wr, 3 op/s 2023-11-17T00:18:00.852 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:00 smithi087 ceph-mon[131182]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:00.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:00 smithi186 ceph-mon[131487]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:00.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:00 smithi006 ceph-mon[97840]: pgmap v282: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:02.865 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:02 smithi186 ceph-mon[131487]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-17T00:18:02.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:02 smithi006 ceph-mon[97840]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-17T00:18:03.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:02 smithi087 ceph-mon[131182]: pgmap v283: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-17T00:18:04.866 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:04 smithi186 ceph-mon[131487]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-17T00:18:04.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:04 smithi006 ceph-mon[97840]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-17T00:18:05.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:04 smithi087 ceph-mon[131182]: pgmap v284: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-17T00:18:06.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:06 smithi006 ceph-mon[97840]: pgmap v285: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-11-17T00:18:07.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:06 smithi087 ceph-mon[131182]: pgmap v285: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-11-17T00:18:07.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:06 smithi186 ceph-mon[131487]: pgmap v285: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-11-17T00:18:07.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:07 smithi006 ceph-mon[97840]: osdmap e100: 12 total, 12 up, 12 in 2023-11-17T00:18:08.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:07 smithi087 ceph-mon[131182]: osdmap e100: 12 total, 12 up, 12 in 2023-11-17T00:18:08.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:07 smithi186 ceph-mon[131487]: osdmap e100: 12 total, 12 up, 12 in 2023-11-17T00:18:08.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:08 smithi006 ceph-mon[97840]: pgmap v287: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:09.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:08 smithi087 ceph-mon[131182]: pgmap v287: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:09.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:08 smithi186 ceph-mon[131487]: pgmap v287: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:10.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:10 smithi006 ceph-mon[97840]: pgmap v288: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:11.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:10 smithi087 ceph-mon[131182]: pgmap v288: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:11.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:10 smithi186 ceph-mon[131487]: pgmap v288: 129 pgs: 129 active+clean; 4.9 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-11-17T00:18:12.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:12 smithi006 ceph-mon[97840]: pgmap v289: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:13.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:12 smithi087 ceph-mon[131182]: pgmap v289: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:13.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:12 smithi186 ceph-mon[131487]: pgmap v289: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:14.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:14 smithi006 ceph-mon[97840]: pgmap v290: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:15.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:14 smithi087 ceph-mon[131182]: pgmap v290: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:15.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:14 smithi186 ceph-mon[131487]: pgmap v290: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 28 KiB/s wr, 5 op/s 2023-11-17T00:18:16.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:16 smithi006 ceph-mon[97840]: pgmap v291: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:18:17.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:16 smithi087 ceph-mon[131182]: pgmap v291: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:18:17.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:16 smithi186 ceph-mon[131487]: pgmap v291: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:18:18.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:18 smithi006 ceph-mon[97840]: pgmap v292: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:18:19.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:18 smithi087 ceph-mon[131182]: pgmap v292: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:18:19.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:18 smithi186 ceph-mon[131487]: pgmap v292: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:18:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:20 smithi006 ceph-mon[97840]: pgmap v293: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:18:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:20 smithi087 ceph-mon[131182]: pgmap v293: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:18:21.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:20 smithi186 ceph-mon[131487]: pgmap v293: 129 pgs: 72 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2023-11-17T00:18:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:22 smithi006 ceph-mon[97840]: pgmap v294: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:18:23.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:22 smithi087 ceph-mon[131182]: pgmap v294: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:18:23.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:22 smithi186 ceph-mon[131487]: pgmap v294: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:18:23.244 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 -- 172.21.15.6:0/3321211346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc1121b0 msgr2=0x7fe8cc114590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/3321211346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc114590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009990 tx=0x7fe8bc009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 -- 172.21.15.6:0/3321211346 shutdown_connections 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/3321211346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc114590 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/3321211346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc108c00 0x7fe8cc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/3321211346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc109510 0x7fe8cc111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.623+0000 7fe8d3bff700 1 -- 172.21.15.6:0/3321211346 >> 172.21.15.6:0/3321211346 conn(0x7fe8cc06c9f0 msgr2=0x7fe8cc071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.624+0000 7fe8d3bff700 1 -- 172.21.15.6:0/3321211346 shutdown_connections 2023-11-17T00:18:23.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.624+0000 7fe8d3bff700 1 -- 172.21.15.6:0/3321211346 wait complete. 2023-11-17T00:18:23.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.626+0000 7fe8d3bff700 1 Processor -- start 2023-11-17T00:18:23.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.626+0000 7fe8d3bff700 1 -- start start 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc108c00 0x7fe8cc1150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc1acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc115ea0 con 0x7fe8cc1121b0 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc1ad1a0 con 0x7fe8cc109510 2023-11-17T00:18:23.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d3bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc1ad2e0 con 0x7fe8cc108c00 2023-11-17T00:18:23.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d119a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d119a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59542/0 (socket says 172.21.15.6:59542) 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.627+0000 7fe8d119a700 1 -- 172.21.15.6:0/1895997697 learned_addr learned my addr 172.21.15.6:0/1895997697 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc1acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d119a700 1 -- 172.21.15.6:0/1895997697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc108c00 msgr2=0x7fe8cc1150c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc108c00 0x7fe8cc1150c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d119a700 1 -- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc1121b0 msgr2=0x7fe8cc1acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc1acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.628+0000 7fe8d119a700 1 -- 172.21.15.6:0/1895997697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0095f0 con 0x7fe8cc109510 2023-11-17T00:18:23.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c930 tx=0x7fe8c800ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8c2ffd700 1 -- 172.21.15.6:0/1895997697 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007ac0 con 0x7fe8cc109510 2023-11-17T00:18:23.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8c2ffd700 1 -- 172.21.15.6:0/1895997697 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c8007c20 con 0x7fe8cc109510 2023-11-17T00:18:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8c2ffd700 1 -- 172.21.15.6:0/1895997697 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8018660 con 0x7fe8cc109510 2023-11-17T00:18:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc118b70 con 0x7fe8cc109510 2023-11-17T00:18:23.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.629+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc1189e0 con 0x7fe8cc109510 2023-11-17T00:18:23.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.637+0000 7fe8c2ffd700 1 -- 172.21.15.6:0/1895997697 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe8c800f450 con 0x7fe8cc109510 2023-11-17T00:18:23.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.637+0000 7fe8e7525b80 1 Processor -- start 2023-11-17T00:18:23.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 -- start start 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4838b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4839510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f4731060 con 0x5645f4834c80 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f47838a0 con 0x5645f48390c0 2023-11-17T00:18:23.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.638+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f477cd30 con 0x5645f4834370 2023-11-17T00:18:23.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.639+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.639+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34306/0 (socket says 172.21.15.6:34306) 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.639+0000 7fe8d219c700 1 -- 172.21.15.6:0/1979723904 learned_addr learned my addr 172.21.15.6:0/1979723904 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.639+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4839510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 -- 172.21.15.6:0/1979723904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 msgr2=0x5645f4839510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4839510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8c2ffd700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 0x7fe8b80a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 -- 172.21.15.6:0/1979723904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 msgr2=0x5645f4838b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4838b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 -- 172.21.15.6:0/1979723904 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f45fe630 con 0x5645f4834370 2023-11-17T00:18:23.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8c2ffd700 1 -- 172.21.15.6:0/1895997697 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe8c80ccfb0 con 0x7fe8cc109510 2023-11-17T00:18:23.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc005cc0 tx=0x7fe8bc009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee551d5a72a39244 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.640+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 0x7fe8b80a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.641+0000 7fe8b7fff700 1 -- 172.21.15.6:0/1979723904 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc02a030 con 0x5645f4834370 2023-11-17T00:18:23.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.641+0000 7fe8b7fff700 1 -- 172.21.15.6:0/1979723904 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc01e040 con 0x5645f4834370 2023-11-17T00:18:23.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.641+0000 7fe8b7fff700 1 -- 172.21.15.6:0/1979723904 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00fbf0 con 0x5645f4834370 2023-11-17T00:18:23.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1979723904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 msgr2=0x5645f4834740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc005cc0 tx=0x7fe8bc009ee0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 0x7fe8b80a38c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc02a000 tx=0x7fe8bc02b000 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1979723904 shutdown_connections 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4839510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4834740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1979723904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4838b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.642+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1979723904 >> 172.21.15.6:0/1979723904 conn(0x5645f4821360 msgr2=0x5645f4826b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.643+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1979723904 shutdown_connections 2023-11-17T00:18:23.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.643+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1979723904 wait complete. 2023-11-17T00:18:23.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.644+0000 7fe8e7525b80 1 -- start start 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.644+0000 7fe8e7525b80 1 Processor -- start 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.645+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.645+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f483da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.645+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 0x5645f483e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f471e8e0 con 0x5645f4834c80 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f471e580 con 0x5645f4834370 2023-11-17T00:18:23.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f4600920 con 0x5645f48390c0 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8e7525b80 10 client.0 _ll_register_callbacks cb 0x5645f473c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8d199b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8d199b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59568/0 (socket says 172.21.15.6:59568) 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8d199b700 1 -- 172.21.15.6:0/2254625420 learned_addr learned my addr 172.21.15.6:0/2254625420 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.646+0000 7fe8d219c700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 0x5645f483e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d219c700 1 -- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 msgr2=0x5645f4840880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d119a700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f483da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d219c700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d219c700 1 -- 172.21.15.6:0/2254625420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 msgr2=0x5645f483da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d219c700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f483da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.647+0000 7fe8d219c700 1 -- 172.21.15.6:0/2254625420 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc107af0 con 0x5645f48390c0 2023-11-17T00:18:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.648+0000 7fe8d199b700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:23.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.648+0000 7fe8d119a700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f483da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.648+0000 7fe8b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.648+0000 7fe8d219c700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 0x5645f483e0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098e0 tx=0x7fe8bc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df272bd80cc5eb79 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.649+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00b9d0 con 0x5645f48390c0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.649+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe8bc02b070 con 0x5645f48390c0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.650+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645f481eb20 con 0x5645f48390c0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.650+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8bc00b9d0 con 0x5645f48390c0 2023-11-17T00:18:23.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.650+0000 7fe8e7525b80 10 client.26932 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.650+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5645f4743190 con 0x5645f48390c0 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.651+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc039920 con 0x5645f48390c0 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.651+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8bc041070 con 0x5645f48390c0 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 10 client.26932 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 20 client.26932 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f484db10 0x5645f484fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 4 client.26932 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:18:23.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645f481eb20 con 0x5645f484db10 2023-11-17T00:18:23.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.652+0000 7fe8d199b700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f484db10 0x5645f484fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.653+0000 7fe8d199b700 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f484db10 0x5645f484fef0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b85d5ffd147f5a server_cookie=420644cccb05d2ef in_seq=0 out_seq=0 2023-11-17T00:18:23.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.653+0000 7fe8b57fa700 10 client.26932 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:23.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.654+0000 7fe8b57fa700 1 -- 172.21.15.6:0/2254625420 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5645f481eb20 con 0x5645f484db10 2023-11-17T00:18:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.655+0000 7fe8b57fa700 10 client.26932 handle_command_reply: tid=0 2023-11-17T00:18:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.656+0000 7fe8e7525b80 1 client.26932 shutdown 2023-11-17T00:18:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.656+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 msgr2=0x5645f483e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.656+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 0x5645f483e0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098e0 tx=0x7fe8bc00bea0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 shutdown_connections 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f48390c0 0x5645f483e0c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834370 0x5645f4840880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f484db10 0x5645f484fef0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/2254625420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f483da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 >> 172.21.15.6:0/2254625420 conn(0x5645f4821360 msgr2=0x5645f483a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8b57fa700 0 client.26932 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 shutdown_connections 2023-11-17T00:18:23.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.660+0000 7fe8e7525b80 1 -- 172.21.15.6:0/2254625420 wait complete. 2023-11-17T00:18:23.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.663+0000 7fe8e7525b80 20 client.26932 trim_cache size 0 max 16384 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.698+0000 7fe8e7525b80 1 Processor -- start 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.698+0000 7fe8e7525b80 1 -- start start 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4845150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4825bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f4820f20 con 0x5645f4834c80 2023-11-17T00:18:23.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f4835050 con 0x5645f48390c0 2023-11-17T00:18:23.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.699+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f4850520 con 0x5645f4834370 2023-11-17T00:18:23.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59574/0 (socket says 172.21.15.6:59574) 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 -- 172.21.15.6:0/1377387743 learned_addr learned my addr 172.21.15.6:0/1377387743 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4845150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 -- 172.21.15.6:0/1377387743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 msgr2=0x5645f4845150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4845150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 -- 172.21.15.6:0/1377387743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 msgr2=0x5645f4825bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4825bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.700+0000 7fe8d219c700 1 -- 172.21.15.6:0/1377387743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f4822b00 con 0x5645f48390c0 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.701+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4845150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.701+0000 7fe8d219c700 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00b5c0 tx=0x7fe8bc0056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240268e52460296e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.701+0000 7fe8c0bff700 1 -- 172.21.15.6:0/1377387743 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc009ee0 con 0x5645f48390c0 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.701+0000 7fe8c0bff700 1 -- 172.21.15.6:0/1377387743 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc005a00 con 0x5645f48390c0 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.701+0000 7fe8c0bff700 1 -- 172.21.15.6:0/1377387743 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc039460 con 0x5645f48390c0 2023-11-17T00:18:23.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.702+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1377387743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 msgr2=0x5645f4826600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.702+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00b5c0 tx=0x7fe8bc0056e0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1377387743 shutdown_connections 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f48390c0 0x5645f4826600 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4845150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1377387743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f4834c80 0x5645f4825bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1377387743 >> 172.21.15.6:0/1377387743 conn(0x5645f4821360 msgr2=0x5645f48370c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.703+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1377387743 shutdown_connections 2023-11-17T00:18:23.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.706+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1377387743 wait complete. 2023-11-17T00:18:23.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.707+0000 7fe8e7525b80 1 -- start start 2023-11-17T00:18:23.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.707+0000 7fe8e7525b80 1 Processor -- start 2023-11-17T00:18:23.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4841130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 0x5645f4841670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 0x5645f4841bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f48a0920 con 0x5645f48390c0 2023-11-17T00:18:23.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f48a0a60 con 0x5645f4834c80 2023-11-17T00:18:23.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f48a0ba0 con 0x5645f4834370 2023-11-17T00:18:23.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.708+0000 7fe8e7525b80 10 client.0 _ll_register_callbacks cb 0x7fe8c80d0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:23.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.709+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 0x5645f4841bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.709+0000 7fe8d219c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 0x5645f4841bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40910/0 (socket says 172.21.15.6:40910) 2023-11-17T00:18:23.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.709+0000 7fe8d219c700 1 -- 172.21.15.6:0/1677080376 learned_addr learned my addr 172.21.15.6:0/1677080376 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:23.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.709+0000 7fe8d119a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 0x5645f4841670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.709+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4841130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d119a700 1 -- 172.21.15.6:0/1677080376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 msgr2=0x5645f4841130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4841130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d119a700 1 -- 172.21.15.6:0/1677080376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 msgr2=0x5645f4841bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:23.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 0x5645f4841bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d119a700 1 -- 172.21.15.6:0/1677080376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645f4743190 con 0x5645f4834c80 2023-11-17T00:18:23.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.710+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4841130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.711+0000 7fe8d119a700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 0x5645f4841670 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80ca300 tx=0x7fe8c80cd170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ebaa3de6406826 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.711+0000 7fe8b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.711+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80997c0 con 0x5645f4834c80 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.711+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe8c8099c80 con 0x5645f4834c80 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.712+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8bc01e040 con 0x5645f4834c80 2023-11-17T00:18:23.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.712+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c80995e0 con 0x5645f4834c80 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.712+0000 7fe8e7525b80 10 client.26975 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.712+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5645f45aade0 con 0x5645f4834c80 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.713+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc01e040 con 0x5645f4834c80 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8c80d0c70 con 0x5645f4834c80 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 10 client.26975 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:23.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 20 client.26975 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f48a21e0 0x5645f48a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 4 client.26975 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.714+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5645f473f0a0 con 0x5645f48a21e0 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.715+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f48a21e0 0x5645f48a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.716+0000 7fe8d199b700 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f48a21e0 0x5645f48a45f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddafdc20927df86d server_cookie=3f0c02c2dee35956 in_seq=0 out_seq=0 2023-11-17T00:18:23.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.716+0000 7fe8b57fa700 10 client.26975 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:23.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.716+0000 7fe8b57fa700 1 -- 172.21.15.6:0/1677080376 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5645f473f0a0 con 0x5645f48a21e0 2023-11-17T00:18:23.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.716+0000 7fe8b57fa700 10 client.26975 handle_command_reply: tid=0 2023-11-17T00:18:23.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.716+0000 7fe8e7525b80 1 client.26975 shutdown 2023-11-17T00:18:23.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 msgr2=0x5645f4841670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 0x5645f4841670 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80ca300 tx=0x7fe8c80cd170 comp rx=0 tx=0).stop 2023-11-17T00:18:23.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 shutdown_connections 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5645f48a21e0 0x5645f48a45f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5645f48390c0 0x5645f4841bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645f4834370 0x5645f4841130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 --2- 172.21.15.6:0/1677080376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5645f4834c80 0x5645f4841670 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 >> 172.21.15.6:0/1677080376 conn(0x5645f4821360 msgr2=0x5645f48370c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8b57fa700 0 client.26975 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:23.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 shutdown_connections 2023-11-17T00:18:23.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.717+0000 7fe8e7525b80 1 -- 172.21.15.6:0/1677080376 wait complete. 2023-11-17T00:18:23.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.718+0000 7fe8e7525b80 20 client.26975 trim_cache size 0 max 16384 2023-11-17T00:18:23.729 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:18:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:18:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:18:23.730 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 msgr2=0x7fe8b80a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 0x7fe8b80a38c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc02a000 tx=0x7fe8bc02b000 comp rx=0 tx=0).stop 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 msgr2=0x7fe8cc115600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c930 tx=0x7fe8c800ccf0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 shutdown_connections 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe8b80a1410 0x7fe8b80a38c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8cc1121b0 0x7fe8cc1acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8cc108c00 0x7fe8cc1150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 --2- 172.21.15.6:0/1895997697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8cc109510 0x7fe8cc115600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:23.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.719+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 >> 172.21.15.6:0/1895997697 conn(0x7fe8cc06c9f0 msgr2=0x7fe8cc109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:23.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.720+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 shutdown_connections 2023-11-17T00:18:23.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:23.720+0000 7fe8d3bff700 1 -- 172.21.15.6:0/1895997697 wait complete. 2023-11-17T00:18:23.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:18:23.754 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:18:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.150+0000 7fc2421ff700 1 -- 172.21.15.6:0/3247589912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c107ec0 msgr2=0x7fc23c108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.150+0000 7fc2421ff700 1 --2- 172.21.15.6:0/3247589912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c107ec0 0x7fc23c108330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc2300099c0 tx=0x7fc230009cd0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 -- 172.21.15.6:0/3247589912 shutdown_connections 2023-11-17T00:18:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 --2- 172.21.15.6:0/3247589912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c119a40 0x7fc23c11be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 --2- 172.21.15.6:0/3247589912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 --2- 172.21.15.6:0/3247589912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c107ec0 0x7fc23c108330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 -- 172.21.15.6:0/3247589912 >> 172.21.15.6:0/3247589912 conn(0x7fc23c06c790 msgr2=0x7fc23c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:24.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 -- 172.21.15.6:0/3247589912 shutdown_connections 2023-11-17T00:18:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.160+0000 7fc2421ff700 1 -- 172.21.15.6:0/3247589912 wait complete. 2023-11-17T00:18:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.162+0000 7fc2421ff700 1 Processor -- start 2023-11-17T00:18:24.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.162+0000 7fc2421ff700 1 -- start start 2023-11-17T00:18:24.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc2421ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c1a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:24.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc2421ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 0x7fc23c1a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:24.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc2421ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:24.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c1a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:24.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c1a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59598/0 (socket says 172.21.15.6:59598) 2023-11-17T00:18:24.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.163+0000 7fc2421ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c1aa250 con 0x7fc23c107ec0 2023-11-17T00:18:24.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc2421ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c1aa3c0 con 0x7fc23c107af0 2023-11-17T00:18:24.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc2421ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c1aa580 con 0x7fc23c119a40 2023-11-17T00:18:24.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:24.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34370/0 (socket says 172.21.15.6:34370) 2023-11-17T00:18:24.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 -- 172.21.15.6:0/1078983969 learned_addr learned my addr 172.21.15.6:0/1078983969 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:24.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23affd700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 0x7fc23c1a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:24.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 -- 172.21.15.6:0/1078983969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 msgr2=0x7fc23c1a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:24.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c1a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.164+0000 7fc23bfff700 1 -- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 msgr2=0x7fc23c1a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:24.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.165+0000 7fc23bfff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 0x7fc23c1a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.165+0000 7fc23bfff700 1 -- 172.21.15.6:0/1078983969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2300095f0 con 0x7fc23c119a40 2023-11-17T00:18:24.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.165+0000 7fc23affd700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 0x7fc23c1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:24.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.166+0000 7fc23bfff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc23400ef90 tx=0x7fc23400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:24.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.166+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234018070 con 0x7fc23c119a40 2023-11-17T00:18:24.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.166+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc23400f040 con 0x7fc23c119a40 2023-11-17T00:18:24.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.166+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c11f260 con 0x7fc23c119a40 2023-11-17T00:18:24.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.167+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234003e10 con 0x7fc23c119a40 2023-11-17T00:18:24.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.167+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c11f0d0 con 0x7fc23c119a40 2023-11-17T00:18:24.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.172+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc234004ad0 con 0x7fc23c119a40 2023-11-17T00:18:24.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.172+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc228005310 con 0x7fc23c119a40 2023-11-17T00:18:24.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.175+0000 7fc238ff9700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 0x7fc2240a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:24.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.175+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fc2340ccbf0 con 0x7fc23c119a40 2023-11-17T00:18:24.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.175+0000 7fc23b7fe700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 0x7fc2240a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:24.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.176+0000 7fc23b7fe700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 0x7fc2240a3830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006fd0 tx=0x7fc22c008040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:24.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.180+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc234095c90 con 0x7fc23c119a40 2023-11-17T00:18:24.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.692+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc228005180 con 0x7fc23c119a40 2023-11-17T00:18:24.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.696+0000 7fc238ff9700 1 -- 172.21.15.6:0/1078983969 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fc2340992b0 con 0x7fc23c119a40 2023-11-17T00:18:24.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:18:24.700 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:18:24.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.700+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 msgr2=0x7fc2240a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.700+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 0x7fc2240a3830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006fd0 tx=0x7fc22c008040 comp rx=0 tx=0).stop 2023-11-17T00:18:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.700+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 msgr2=0x7fc23c1a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.700+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc23400ef90 tx=0x7fc23400c450 comp rx=0 tx=0).stop 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 shutdown_connections 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc2240a1380 0x7fc2240a3830 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006fd0 tx=0x7fc22c008040 comp rx=0 tx=0).stop 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c119a40 0x7fc23c1a9c10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc23c107af0 0x7fc23c1a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 --2- 172.21.15.6:0/1078983969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc23c107ec0 0x7fc23c1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.701+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 >> 172.21.15.6:0/1078983969 conn(0x7fc23c06c790 msgr2=0x7fc23c071710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.702+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 shutdown_connections 2023-11-17T00:18:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:24.702+0000 7fc2421ff700 1 -- 172.21.15.6:0/1078983969 wait complete. 2023-11-17T00:18:24.706 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:18:24.740 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:18:25.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:24 smithi087 ceph-mon[131182]: pgmap v295: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:18:25.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:24 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:18:25.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:24 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1078983969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:18:25.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:24 smithi186 ceph-mon[131487]: pgmap v295: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:18:25.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:24 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:18:25.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:24 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1078983969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:18:25.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.186+0000 7fe7063ff700 1 -- 172.21.15.6:0/1567356558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700110f70 msgr2=0x7fe7001133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:25.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.186+0000 7fe7063ff700 1 --2- 172.21.15.6:0/1567356558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700110f70 0x7fe7001133b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800b0a0 tx=0x7fe6f800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 -- 172.21.15.6:0/1567356558 shutdown_connections 2023-11-17T00:18:25.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 --2- 172.21.15.6:0/1567356558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe700068030 0x7fe700068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 --2- 172.21.15.6:0/1567356558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700110f70 0x7fe7001133b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 --2- 172.21.15.6:0/1567356558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700068940 0x7fe7001108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 -- 172.21.15.6:0/1567356558 >> 172.21.15.6:0/1567356558 conn(0x7fe7000763a0 msgr2=0x7fe7000767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 -- 172.21.15.6:0/1567356558 shutdown_connections 2023-11-17T00:18:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.187+0000 7fe7063ff700 1 -- 172.21.15.6:0/1567356558 wait complete. 2023-11-17T00:18:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.189+0000 7fe7063ff700 1 Processor -- start 2023-11-17T00:18:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.190+0000 7fe7063ff700 1 -- start start 2023-11-17T00:18:25.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe7063ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe700068030 0x7fe7001a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:25.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe7063ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700068940 0x7fe7001a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:25.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe7063ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:25.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe7063ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70019cf80 con 0x7fe700068030 2023-11-17T00:18:25.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe7063ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70019d0f0 con 0x7fe700110f70 2023-11-17T00:18:25.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe70499c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:25.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe70499c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59630/0 (socket says 172.21.15.6:59630) 2023-11-17T00:18:25.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.191+0000 7fe70499c700 1 -- 172.21.15.6:0/2226293560 learned_addr learned my addr 172.21.15.6:0/2226293560 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70019d2b0 con 0x7fe700068940 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe70499c700 1 -- 172.21.15.6:0/2226293560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700068940 msgr2=0x7fe7001a2de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe70499c700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700068940 0x7fe7001a2de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe70499c700 1 -- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe700068030 msgr2=0x7fe7001a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe70499c700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe700068030 0x7fe7001a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:25.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.192+0000 7fe70499c700 1 -- 172.21.15.6:0/2226293560 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f8009d00 con 0x7fe700110f70 2023-11-17T00:18:25.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.193+0000 7fe70499c700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800b070 tx=0x7fe6f8005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:25.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.194+0000 7fe6fd7fa700 1 -- 172.21.15.6:0/2226293560 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800e040 con 0x7fe700110f70 2023-11-17T00:18:25.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.194+0000 7fe6fd7fa700 1 -- 172.21.15.6:0/2226293560 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f8012e40 con 0x7fe700110f70 2023-11-17T00:18:25.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.194+0000 7fe6fd7fa700 1 -- 172.21.15.6:0/2226293560 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f801b450 con 0x7fe700110f70 2023-11-17T00:18:25.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.194+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe700116ac0 con 0x7fe700110f70 2023-11-17T00:18:25.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.194+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe700116930 con 0x7fe700110f70 2023-11-17T00:18:25.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.208+0000 7fe719d1db80 1 Processor -- start 2023-11-17T00:18:25.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.209+0000 7fe719d1db80 1 -- start start 2023-11-17T00:18:25.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.209+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:25.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.209+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:25.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.209+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a7f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe719d1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a6f8060 con 0x559d3a7f2b60 2023-11-17T00:18:26.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe719d1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a74a8a0 con 0x559d3a7f78b0 2023-11-17T00:18:26.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe719d1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a743d30 con 0x559d3a7f3470 2023-11-17T00:18:26.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe6fd7fa700 1 -- 172.21.15.6:0/2226293560 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe6f801d6a0 con 0x7fe700110f70 2023-11-17T00:18:26.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a7f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34400/0 (socket says 172.21.15.6:34400) 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.210+0000 7fe6fffff700 1 -- 172.21.15.6:0/3550337908 learned_addr learned my addr 172.21.15.6:0/3550337908 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.211+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.212+0000 7fe6fd7fa700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 0x7fe6e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.212+0000 7fe6fd7fa700 1 -- 172.21.15.6:0/2226293560 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe6f80ceec0 con 0x7fe700110f70 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.212+0000 7fe6fffff700 1 -- 172.21.15.6:0/3550337908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 msgr2=0x559d3a7f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.212+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a7f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.138 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.212+0000 7fe6fffff700 1 -- 172.21.15.6:0/3550337908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 msgr2=0x559d3a7f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.214+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.214+0000 7fe6fffff700 1 -- 172.21.15.6:0/3550337908 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3a5c5630 con 0x559d3a7f3470 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.214+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.214+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40098a0 tx=0x7fe6f4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d206e8cf3f24e268 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.214+0000 7fe6e67fc700 1 -- 172.21.15.6:0/3550337908 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400b870 con 0x559d3a7f3470 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.215+0000 7fe6e67fc700 1 -- 172.21.15.6:0/3550337908 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f400be90 con 0x559d3a7f3470 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.215+0000 7fe6e67fc700 1 -- 172.21.15.6:0/3550337908 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400b870 con 0x559d3a7f3470 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.216+0000 7fe719d1db80 1 -- 172.21.15.6:0/3550337908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 msgr2=0x559d3a7f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.216+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40098a0 tx=0x7fe6f4009bb0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.216+0000 7fe70499c700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 0x7fe6e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 -- 172.21.15.6:0/3550337908 shutdown_connections 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a7f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3550337908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f7370 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 -- 172.21.15.6:0/3550337908 >> 172.21.15.6:0/3550337908 conn(0x559d3a7e8360 msgr2=0x559d3a7edb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 -- 172.21.15.6:0/3550337908 shutdown_connections 2023-11-17T00:18:26.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.217+0000 7fe719d1db80 1 -- 172.21.15.6:0/3550337908 wait complete. 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.218+0000 7fe719d1db80 1 -- start start 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.218+0000 7fe719d1db80 1 Processor -- start 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.219+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 0x559d3a7fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 0x559d3a7fc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 0x559d3a7fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 0x559d3a7fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59646/0 (socket says 172.21.15.6:59646) 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe6fffff700 1 -- 172.21.15.6:0/2433854127 learned_addr learned my addr 172.21.15.6:0/2433854127 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:26.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a6e58e0 con 0x559d3a7f2b60 2023-11-17T00:18:26.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a6e5580 con 0x559d3a7f3470 2023-11-17T00:18:26.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a5c7920 con 0x559d3a7f78b0 2023-11-17T00:18:26.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.220+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 0x559d3a7fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.221+0000 7fe719d1db80 10 client.0 _ll_register_callbacks cb 0x559d3a703780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:26.142 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:24 smithi006 ceph-mon[97840]: pgmap v295: 129 pgs: 70 active+clean+snaptrim_wait, 36 active+clean, 23 active+clean+snaptrim; 4.9 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:18:26.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:24 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:18:26.175 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:24 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1078983969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:18:26.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.219+0000 7fe70499c700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 0x7fe6e80a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8098630 tx=0x7fe6f8009040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:26.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 -- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 msgr2=0x559d3a7fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 0x559d3a7fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 -- 172.21.15.6:0/2433854127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 msgr2=0x559d3a7ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40094d0 con 0x559d3a7f78b0 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.222+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 0x559d3a7fc870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe700077730 tx=0x7fe6f000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e380b4f09fcf1a84 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe6df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0004020 con 0x559d3a7f78b0 2023-11-17T00:18:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe6f0004180 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3a7e5b20 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f0004340 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe719d1db80 10 client.26953 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.224+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559d3a70a190 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d3a7e5b20 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6f0011820 con 0x559d3a7f78b0 2023-11-17T00:18:26.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 10 client.26953 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 20 client.26953 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a80c8b0 0x559d3a80ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 4 client.26953 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.225+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d3a7060a0 con 0x559d3a80c8b0 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.226+0000 7fe6fffff700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a80c8b0 0x559d3a80ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.227+0000 7fe6df7fe700 10 client.26953 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:26.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.227+0000 7fe6fffff700 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a80c8b0 0x559d3a80ecc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a29d40d57a12b6 server_cookie=ecc9827f100a7abb in_seq=0 out_seq=0 2023-11-17T00:18:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.227+0000 7fe6df7fe700 1 -- 172.21.15.6:0/2433854127 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559d3a7060a0 con 0x559d3a80c8b0 2023-11-17T00:18:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.227+0000 7fe6df7fe700 10 client.26953 handle_command_reply: tid=0 2023-11-17T00:18:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.228+0000 7fe719d1db80 1 client.26953 shutdown 2023-11-17T00:18:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.229+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 msgr2=0x559d3a7fc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.229+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 0x559d3a7fc870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe700077730 tx=0x7fe6f000ef90 comp rx=0 tx=0).stop 2023-11-17T00:18:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 shutdown_connections 2023-11-17T00:18:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a80c8b0 0x559d3a80ecc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a7ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f78b0 0x559d3a7fc870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 --2- 172.21.15.6:0/2433854127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f3470 0x559d3a7fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.231+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 >> 172.21.15.6:0/2433854127 conn(0x559d3a7e8360 msgr2=0x559d3a7f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.233+0000 7fe6df7fe700 0 client.26953 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.233+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 shutdown_connections 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.233+0000 7fe719d1db80 1 -- 172.21.15.6:0/2433854127 wait complete. 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.233+0000 7fe719d1db80 20 client.26953 trim_cache size 0 max 16384 2023-11-17T00:18:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.261+0000 7fe719d1db80 1 Processor -- start 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.261+0000 7fe719d1db80 1 -- start start 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f2b60 0x559d3a8039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f78b0 0x559d3a7feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a7e7f20 con 0x559d3a7f78b0 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a7f6390 con 0x559d3a7f2b60 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe719d1db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a7f3900 con 0x559d3a7f3470 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34418/0 (socket says 172.21.15.6:34418) 2023-11-17T00:18:26.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe6fffff700 1 -- 172.21.15.6:0/3260543891 learned_addr learned my addr 172.21.15.6:0/3260543891 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:26.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.262+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f2b60 0x559d3a8039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 -- 172.21.15.6:0/3260543891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f2b60 msgr2=0x559d3a8039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f2b60 0x559d3a8039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 -- 172.21.15.6:0/3260543891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f78b0 msgr2=0x559d3a7feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f78b0 0x559d3a7feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 -- 172.21.15.6:0/3260543891 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3a709ee0 con 0x559d3a7f3470 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6fffff700 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4000c00 tx=0x7fe6f4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e2864a523451c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6e6ffd700 1 -- 172.21.15.6:0/3260543891 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4031070 con 0x559d3a7f3470 2023-11-17T00:18:26.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6e6ffd700 1 -- 172.21.15.6:0/3260543891 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f4035410 con 0x559d3a7f3470 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.263+0000 7fe6e6ffd700 1 -- 172.21.15.6:0/3260543891 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4035780 con 0x559d3a7f3470 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.264+0000 7fe719d1db80 1 -- 172.21.15.6:0/3260543891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 msgr2=0x559d3a7f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.264+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4000c00 tx=0x7fe6f4009800 comp rx=0 tx=0).stop 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 -- 172.21.15.6:0/3260543891 shutdown_connections 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f78b0 0x559d3a7feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f2b60 0x559d3a8039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 --2- 172.21.15.6:0/3260543891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a7f5970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 -- 172.21.15.6:0/3260543891 >> 172.21.15.6:0/3260543891 conn(0x559d3a7e8360 msgr2=0x559d3a7f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 -- 172.21.15.6:0/3260543891 shutdown_connections 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.265+0000 7fe719d1db80 1 -- 172.21.15.6:0/3260543891 wait complete. 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.266+0000 7fe719d1db80 1 -- start start 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.267+0000 7fe719d1db80 1 Processor -- start 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a802b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe719d1db80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a85f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34430/0 (socket says 172.21.15.6:34430) 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.268+0000 7fe6fffff700 1 -- 172.21.15.6:0/1624507747 learned_addr learned my addr 172.21.15.6:0/1624507747 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a803810 con 0x559d3a7f2b60 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a803980 con 0x559d3a7f78b0 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3a85fc70 con 0x559d3a7f3470 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a85f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe719d1db80 10 client.0 _ll_register_callbacks cb 0x559d3a703780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:26.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a802b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6fffff700 1 -- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 msgr2=0x559d3a85f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6fffff700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a85f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6fffff700 1 -- 172.21.15.6:0/1624507747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 msgr2=0x559d3a802b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6fffff700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a802b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6fffff700 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d3a5c5630 con 0x559d3a7f3470 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.269+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a802b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.270+0000 7fe6fffff700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40355c0 tx=0x7fe6f40356a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265f52f95f0411a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:26.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.270+0000 7fe6df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:18:26.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.273+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4031050 con 0x559d3a7f3470 2023-11-17T00:18:26.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.273+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe6f40116a0 con 0x559d3a7f3470 2023-11-17T00:18:26.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.273+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f400be90 con 0x559d3a7f3470 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.274+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f4011860 con 0x559d3a7f3470 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.274+0000 7fe719d1db80 10 client.26959 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.274+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559d3a55ac60 con 0x559d3a7f3470 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.275+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f400be90 con 0x559d3a7f3470 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6f4056050 con 0x559d3a7f3470 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 10 client.26959 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:26.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:26.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 20 client.26959 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:26.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a860e70 0x559d3a863250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:26.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 4 client.26959 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:18:26.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d3a712660 con 0x559d3a860e70 2023-11-17T00:18:26.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.278+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a860e70 0x559d3a863250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:26.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.280+0000 7fe6df7fe700 10 client.26959 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:26.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.280+0000 7fe6ff7fe700 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a860e70 0x559d3a863250 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596fa9767e000217 server_cookie=4a23120b60606c78 in_seq=0 out_seq=0 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.280+0000 7fe6df7fe700 1 -- 172.21.15.6:0/1624507747 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d3a712660 con 0x559d3a860e70 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.281+0000 7fe6df7fe700 10 client.26959 handle_command_reply: tid=0 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.281+0000 7fe719d1db80 1 client.26959 shutdown 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.282+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 msgr2=0x559d3a8030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.282+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40355c0 tx=0x7fe6f40356a0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.282+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 shutdown_connections 2023-11-17T00:18:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.282+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x559d3a860e70 0x559d3a863250 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.283+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559d3a7f2b60 0x559d3a802b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.283+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x559d3a7f78b0 0x559d3a85f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.283+0000 7fe719d1db80 1 --2- 172.21.15.6:0/1624507747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d3a7f3470 0x559d3a8030a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.283+0000 7fe6df7fe700 0 client.26959 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.283+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 >> 172.21.15.6:0/1624507747 conn(0x559d3a7e8360 msgr2=0x559d3a7f5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.284+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 shutdown_connections 2023-11-17T00:18:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.284+0000 7fe719d1db80 1 -- 172.21.15.6:0/1624507747 wait complete. 2023-11-17T00:18:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.288+0000 7fe719d1db80 20 client.26959 trim_cache size 0 max 16384 2023-11-17T00:18:26.623 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:18:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.289+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 msgr2=0x7fe6e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.289+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 0x7fe6e80a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8098630 tx=0x7fe6f8009040 comp rx=0 tx=0).stop 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.289+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 msgr2=0x7fe70019c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.289+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800b070 tx=0x7fe6f8005a60 comp rx=0 tx=0).stop 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 shutdown_connections 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fe6e80a1490 0x7fe6e80a3940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe700068030 0x7fe7001a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe700110f70 0x7fe70019c910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 --2- 172.21.15.6:0/2226293560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe700068940 0x7fe7001a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 >> 172.21.15.6:0/2226293560 conn(0x7fe7000763a0 msgr2=0x7fe70010e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 shutdown_connections 2023-11-17T00:18:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:25.291+0000 7fe7063ff700 1 -- 172.21.15.6:0/2226293560 wait complete. 2023-11-17T00:18:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: pgmap v296: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:18:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:18:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:18:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:18:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:18:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:18:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:26 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: pgmap v296: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:18:27.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:26 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: pgmap v296: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:18:27.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:26 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:18:27.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-17T00:18:27.627 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-17T00:18:28.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 -- 172.21.15.6:0/280305214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06b090 msgr2=0x7f07ac06f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 --2- 172.21.15.6:0/280305214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06b090 0x7f07ac06f9b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f07940098f0 tx=0x7f0794009c00 comp rx=0 tx=0).stop 2023-11-17T00:18:28.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 -- 172.21.15.6:0/280305214 shutdown_connections 2023-11-17T00:18:28.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 --2- 172.21.15.6:0/280305214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07ac06ffd0 0x7f07ac070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 --2- 172.21.15.6:0/280305214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06a780 0x7f07ac06ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 --2- 172.21.15.6:0/280305214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06b090 0x7f07ac06f9b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 -- 172.21.15.6:0/280305214 >> 172.21.15.6:0/280305214 conn(0x7f07ac0763a0 msgr2=0x7f07ac0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 -- 172.21.15.6:0/280305214 shutdown_connections 2023-11-17T00:18:28.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.007+0000 7f07b2dff700 1 -- 172.21.15.6:0/280305214 wait complete. 2023-11-17T00:18:28.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.009+0000 7f07b2dff700 1 Processor -- start 2023-11-17T00:18:28.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.010+0000 7f07b2dff700 1 -- start start 2023-11-17T00:18:28.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.010+0000 7f07b2dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 0x7f07ac1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b2dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b2dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07ac06ffd0 0x7f07ac1a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b2dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1a5e00 con 0x7f07ac06a780 2023-11-17T00:18:28.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b2dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1a5f70 con 0x7f07ac06ffd0 2023-11-17T00:18:28.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b2dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1a6130 con 0x7f07ac06b090 2023-11-17T00:18:28.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07b0b9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 0x7f07ac1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34460/0 (socket says 172.21.15.6:34460) 2023-11-17T00:18:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.012+0000 7f07a3fff700 1 -- 172.21.15.6:0/1678929393 learned_addr learned my addr 172.21.15.6:0/1678929393 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 -- 172.21.15.6:0/1678929393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07ac06ffd0 msgr2=0x7f07ac1a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07ac06ffd0 0x7f07ac1a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 -- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 msgr2=0x7f07ac1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 0x7f07ac1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 -- 172.21.15.6:0/1678929393 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009520 con 0x7f07ac06b090 2023-11-17T00:18:28.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 0x7f07ac1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:28.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.013+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0794009230 tx=0x7f079400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.014+0000 7f07aa7fc700 1 -- 172.21.15.6:0/1678929393 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079401c070 con 0x7f07ac06b090 2023-11-17T00:18:28.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.014+0000 7f07aa7fc700 1 -- 172.21.15.6:0/1678929393 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079400bb80 con 0x7f07ac06b090 2023-11-17T00:18:28.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.014+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac118c60 con 0x7f07ac06b090 2023-11-17T00:18:28.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.014+0000 7f07aa7fc700 1 -- 172.21.15.6:0/1678929393 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07940049c0 con 0x7f07ac06b090 2023-11-17T00:18:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.014+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac118ad0 con 0x7f07ac06b090 2023-11-17T00:18:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.022+0000 7f07aa7fc700 1 -- 172.21.15.6:0/1678929393 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0794020070 con 0x7f07ac06b090 2023-11-17T00:18:28.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.023+0000 7f07c67f0b80 1 Processor -- start 2023-11-17T00:18:28.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 -- start start 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581515d00e0 con 0x5581516cabe0 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x558151622920 con 0x5581516cf930 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815161bdb0 con 0x5581516cb4f0 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.024+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.025+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59698/0 (socket says 172.21.15.6:59698) 2023-11-17T00:18:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.025+0000 7f07a3fff700 1 -- 172.21.15.6:0/1750973089 learned_addr learned my addr 172.21.15.6:0/1750973089 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07aa7fc700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 0x7f07980a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07a3fff700 1 -- 172.21.15.6:0/1750973089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 msgr2=0x5581516cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07a3fff700 1 -- 172.21.15.6:0/1750973089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 msgr2=0x5581516cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.026+0000 7f07a3fff700 1 -- 172.21.15.6:0/1750973089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55815149d6b0 con 0x5581516cf930 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.027+0000 7f07b139c700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 0x7f07980a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.027+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f07940cb040 tx=0x7f0794096c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95aa2dad58ca428 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.027+0000 7f07a2ffd700 1 -- 172.21.15.6:0/1750973089 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079409a880 con 0x5581516cf930 2023-11-17T00:18:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.027+0000 7f07aa7fc700 1 -- 172.21.15.6:0/1678929393 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f07940cdd50 con 0x7f07ac06b090 2023-11-17T00:18:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.028+0000 7f07a2ffd700 1 -- 172.21.15.6:0/1750973089 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07940d1370 con 0x5581516cf930 2023-11-17T00:18:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.028+0000 7f07a2ffd700 1 -- 172.21.15.6:0/1750973089 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079409a880 con 0x5581516cf930 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.029+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1750973089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 msgr2=0x5581516cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.029+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f07940cb040 tx=0x7f0794096c20 comp rx=0 tx=0).stop 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07b139c700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 0x7f07980a3830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07a40094f0 tx=0x7f07a4009440 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1750973089 shutdown_connections 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516cfd80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1750973089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1750973089 >> 172.21.15.6:0/1750973089 conn(0x5581516c03e0 msgr2=0x5581516c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1750973089 shutdown_connections 2023-11-17T00:18:28.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.030+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1750973089 wait complete. 2023-11-17T00:18:28.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.031+0000 7f07c67f0b80 1 -- start start 2023-11-17T00:18:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.032+0000 7f07c67f0b80 1 Processor -- start 2023-11-17T00:18:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581516d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cf930 0x5581516d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581515bd960 con 0x5581516cb4f0 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581515bd600 con 0x5581516cabe0 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815149f9a0 con 0x5581516cf930 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07c67f0b80 10 client.0 _ll_register_callbacks cb 0x5581515db800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41034/0 (socket says 172.21.15.6:41034) 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.033+0000 7f07a3fff700 1 -- 172.21.15.6:0/3326758607 learned_addr learned my addr 172.21.15.6:0/3326758607 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 -- 172.21.15.6:0/3326758607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cf930 msgr2=0x5581516d4850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cf930 0x5581516d4850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 -- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 msgr2=0x5581516d7050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:18:28.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581516d7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 -- 172.21.15.6:0/3326758607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac118ad0 con 0x5581516cb4f0 2023-11-17T00:18:28.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f07a3fff700 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f07940cb150 tx=0x7f079409a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871ee0a3dc04464 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.034+0000 7f078bfff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-17T00:18:28.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794005440 con 0x5581516cb4f0 2023-11-17T00:18:28.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0794005040 con 0x5581516cb4f0 2023-11-17T00:18:28.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581515de120 con 0x5581516cb4f0 2023-11-17T00:18:28.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079409a5f0 con 0x5581516cb4f0 2023-11-17T00:18:28.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f07c67f0b80 10 client.16842 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:28.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.035+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5581515e2210 con 0x5581516cb4f0 2023-11-17T00:18:28.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581515de120 con 0x5581516cb4f0 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07940d1030 con 0x5581516cb4f0 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 20 client.16842 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5581516e4170 0x5581516e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 4 client.16842 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:18:28.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.036+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581515ea6e0 con 0x5581516e4170 2023-11-17T00:18:28.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.037+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5581516e4170 0x5581516e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.037+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5581516e4170 0x5581516e6550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a449c88f3ef7dc0b server_cookie=29cbbea7d7758116 in_seq=0 out_seq=0 2023-11-17T00:18:28.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.037+0000 7f078bfff700 10 client.16842 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.038+0000 7f078bfff700 1 -- 172.21.15.6:0/3326758607 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5581515ea6e0 con 0x5581516e4170 2023-11-17T00:18:28.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.038+0000 7f078bfff700 10 client.16842 handle_command_reply: tid=0 2023-11-17T00:18:28.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.039+0000 7f07c67f0b80 1 client.16842 shutdown 2023-11-17T00:18:28.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 msgr2=0x5581516d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f07940cb150 tx=0x7f079409a410 comp rx=0 tx=0).stop 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 shutdown_connections 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cf930 0x5581516d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581516d7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5581516e4170 0x5581516e6550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/3326758607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cb4f0 0x5581516d42d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 >> 172.21.15.6:0/3326758607 conn(0x5581516c03e0 msgr2=0x5581516d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f078bfff700 0 client.16842 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.040+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 shutdown_connections 2023-11-17T00:18:28.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.041+0000 7f07c67f0b80 1 -- 172.21.15.6:0/3326758607 wait complete. 2023-11-17T00:18:28.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.041+0000 7f07c67f0b80 20 client.16842 trim_cache size 0 max 16384 2023-11-17T00:18:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.083+0000 7f07c67f0b80 1 Processor -- start 2023-11-17T00:18:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.083+0000 7f07c67f0b80 1 -- start start 2023-11-17T00:18:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581516bff60 con 0x5581516cabe0 2023-11-17T00:18:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581516cb980 con 0x5581516cf930 2023-11-17T00:18:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581515ea6e0 con 0x5581516cb4f0 2023-11-17T00:18:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.085+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34484/0 (socket says 172.21.15.6:34484) 2023-11-17T00:18:28.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07a3fff700 1 -- 172.21.15.6:0/4274017841 learned_addr learned my addr 172.21.15.6:0/4274017841 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07b0b9b700 1 -- 172.21.15.6:0/4274017841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 msgr2=0x5581516d6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07b139c700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.086+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07b0b9b700 1 -- 172.21.15.6:0/4274017841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 msgr2=0x5581516d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07b0b9b700 1 -- 172.21.15.6:0/4274017841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581515e1f60 con 0x5581516cabe0 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07a3fff700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07b139c700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.087+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516d5be0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0794098e00 tx=0x7f079c009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d648b86aa7677c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.088+0000 7f07a37fe700 1 -- 172.21.15.6:0/4274017841 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c004b40 con 0x5581516cabe0 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.088+0000 7f07a37fe700 1 -- 172.21.15.6:0/4274017841 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c00f040 con 0x5581516cabe0 2023-11-17T00:18:28.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.088+0000 7f07a37fe700 1 -- 172.21.15.6:0/4274017841 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c008de0 con 0x5581516cabe0 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.089+0000 7f07c67f0b80 1 -- 172.21.15.6:0/4274017841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 msgr2=0x5581516d5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.089+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516d5be0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0794098e00 tx=0x7f079c009320 comp rx=0 tx=0).stop 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.090+0000 7f07c67f0b80 1 -- 172.21.15.6:0/4274017841 shutdown_connections 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.090+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cabe0 0x5581516d5be0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.090+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cf930 0x5581516d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.090+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/4274017841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x5581516d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.090+0000 7f07c67f0b80 1 -- 172.21.15.6:0/4274017841 >> 172.21.15.6:0/4274017841 conn(0x5581516c03e0 msgr2=0x5581516ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.095+0000 7f07c67f0b80 1 -- 172.21.15.6:0/4274017841 shutdown_connections 2023-11-17T00:18:28.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.095+0000 7f07c67f0b80 1 -- 172.21.15.6:0/4274017841 wait complete. 2023-11-17T00:18:28.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.096+0000 7f07c67f0b80 1 -- start start 2023-11-17T00:18:28.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.097+0000 7f07c67f0b80 1 Processor -- start 2023-11-17T00:18:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.097+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.097+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x558151731ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07c67f0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cf930 0x55815172bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815172c520 con 0x5581516cf930 2023-11-17T00:18:28.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815172c690 con 0x5581516cabe0 2023-11-17T00:18:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07c67f0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815172c850 con 0x5581516cb4f0 2023-11-17T00:18:28.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07c67f0b80 10 client.0 _ll_register_callbacks cb 0x7f0794096e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:28.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07a3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x558151731ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59728/0 (socket says 172.21.15.6:59728) 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 -- 172.21.15.6:0/1704466379 learned_addr learned my addr 172.21.15.6:0/1704466379 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 -- 172.21.15.6:0/1704466379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 msgr2=0x558151731ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x558151731ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 -- 172.21.15.6:0/1704466379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cf930 msgr2=0x55815172bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cf930 0x55815172bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.098+0000 7f07b0b9b700 1 -- 172.21.15.6:0/1704466379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c01bb40 con 0x5581516cabe0 2023-11-17T00:18:28.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.099+0000 7f07b0b9b700 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f079c009bd0 tx=0x7f079c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cb69822dcdb40a server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.099+0000 7f078bfff700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:18:28.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.099+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c01b360 con 0x5581516cabe0 2023-11-17T00:18:28.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f079c01b4c0 con 0x5581516cabe0 2023-11-17T00:18:28.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581516bdba0 con 0x5581516cabe0 2023-11-17T00:18:28.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079c008a20 con 0x5581516cabe0 2023-11-17T00:18:28.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f07c67f0b80 10 client.26999 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:28.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558151432ce0 con 0x5581516cabe0 2023-11-17T00:18:28.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.100+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581516bdba0 con 0x5581516cabe0 2023-11-17T00:18:28.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f079c00ec50 con 0x5581516cabe0 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 10 client.26999 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 10 client.26999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 20 client.26999 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558151741e50 0x558151744260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 4 client.26999 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55815146aff0 con 0x558151741e50 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.102+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558151741e50 0x558151744260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.103+0000 7f07a3fff700 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558151741e50 0x558151744260 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eab426f80b67e1 server_cookie=ebd61b873bfdce11 in_seq=0 out_seq=0 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.103+0000 7f078bfff700 10 client.26999 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.104+0000 7f078bfff700 1 -- 172.21.15.6:0/1704466379 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55815146aff0 con 0x558151741e50 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.104+0000 7f078bfff700 10 client.26999 handle_command_reply: tid=0 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.104+0000 7f07c67f0b80 1 client.26999 shutdown 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 msgr2=0x5581517319a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f079c009bd0 tx=0x7f079c009800 comp rx=0 tx=0).stop 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 shutdown_connections 2023-11-17T00:18:28.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x558151741e50 0x558151744260 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5581516cf930 0x55815172bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5581516cabe0 0x5581517319a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 --2- 172.21.15.6:0/1704466379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5581516cb4f0 0x558151731ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f078bfff700 0 client.26999 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 >> 172.21.15.6:0/1704466379 conn(0x5581516c03e0 msgr2=0x5581516ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 shutdown_connections 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.105+0000 7f07c67f0b80 1 -- 172.21.15.6:0/1704466379 wait complete. 2023-11-17T00:18:28.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.106+0000 7f07c67f0b80 20 client.26999 trim_cache size 0 max 16384 2023-11-17T00:18:28.112 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:18:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-17T00:18:28.112 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "17edca36-76c0-4085-8753-3691a0a6e07f", 2023-11-17T00:18:28.113 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-17T00:18:28.113 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:18:28.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.107+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 msgr2=0x7f07980a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.107+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 0x7f07980a3830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07a40094f0 tx=0x7f07a4009440 comp rx=0 tx=0).stop 2023-11-17T00:18:28.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.107+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 msgr2=0x7f07ac1a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.107+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0794009230 tx=0x7f079400fa00 comp rx=0 tx=0).stop 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 shutdown_connections 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f07980a1380 0x7f07980a3830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07ac06a780 0x7f07ac1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07ac06ffd0 0x7f07ac1a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 --2- 172.21.15.6:0/1678929393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07ac06b090 0x7f07ac1a1360 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 >> 172.21.15.6:0/1678929393 conn(0x7f07ac0763a0 msgr2=0x7f07ac0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.108+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 shutdown_connections 2023-11-17T00:18:28.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.109+0000 7f07b2dff700 1 -- 172.21.15.6:0/1678929393 wait complete. 2023-11-17T00:18:28.144 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:18:28.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.521+0000 7fc061dff700 1 -- 172.21.15.6:0/3904888653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c06dfe0 msgr2=0x7fc05c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.521+0000 7fc061dff700 1 --2- 172.21.15.6:0/3904888653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c06dfe0 0x7fc05c107af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009990 tx=0x7fc04c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 -- 172.21.15.6:0/3904888653 shutdown_connections 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 --2- 172.21.15.6:0/3904888653 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c117640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 --2- 172.21.15.6:0/3904888653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c06dfe0 0x7fc05c107af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 --2- 172.21.15.6:0/3904888653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c108030 0x7fc05c1084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 -- 172.21.15.6:0/3904888653 >> 172.21.15.6:0/3904888653 conn(0x7fc05c06c9d0 msgr2=0x7fc05c06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.522+0000 7fc061dff700 1 -- 172.21.15.6:0/3904888653 shutdown_connections 2023-11-17T00:18:28.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.523+0000 7fc061dff700 1 -- 172.21.15.6:0/3904888653 wait complete. 2023-11-17T00:18:28.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.525+0000 7fc061dff700 1 Processor -- start 2023-11-17T00:18:28.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.525+0000 7fc061dff700 1 -- start start 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 0x7fc05c1a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c1a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c1a5d80 con 0x7fc05c115260 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c1a5ef0 con 0x7fc05c06dfe0 2023-11-17T00:18:28.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.526+0000 7fc061dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c1a60b0 con 0x7fc05c108030 2023-11-17T00:18:28.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c1a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59740/0 (socket says 172.21.15.6:59740) 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c1a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41064/0 (socket says 172.21.15.6:41064) 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1494883023 learned_addr learned my addr 172.21.15.6:0/1494883023 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05affd700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 0x7fc05c1a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1494883023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 msgr2=0x7fc05c1a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 0x7fc05c1a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 msgr2=0x7fc05c1a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c1a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1494883023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044009520 con 0x7fc05c06dfe0 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.527+0000 7fc05affd700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 0x7fc05c1a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc04c005910 tx=0x7fc04c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc058ff9700 1 -- 172.21.15.6:0/1494883023 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c01d070 con 0x7fc05c06dfe0 2023-11-17T00:18:28.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c0095f0 con 0x7fc05c06dfe0 2023-11-17T00:18:28.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc058ff9700 1 -- 172.21.15.6:0/1494883023 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04c022470 con 0x7fc05c06dfe0 2023-11-17T00:18:28.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc058ff9700 1 -- 172.21.15.6:0/1494883023 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c017650 con 0x7fc05c06dfe0 2023-11-17T00:18:28.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.528+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c11ac50 con 0x7fc05c06dfe0 2023-11-17T00:18:28.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.543+0000 7fc058ff9700 1 -- 172.21.15.6:0/1494883023 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc04c00fad0 con 0x7fc05c06dfe0 2023-11-17T00:18:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.548+0000 7fc0757e0b80 1 Processor -- start 2023-11-17T00:18:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.548+0000 7fc0757e0b80 1 -- start start 2023-11-17T00:18:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b235aa060 con 0x555b236a5470 2023-11-17T00:18:28.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b235fc8a0 con 0x555b236a98b0 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b235f5d30 con 0x555b236a4b60 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc058ff9700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 0x7fc0480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc05affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc05affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59742/0 (socket says 172.21.15.6:59742) 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc05affd700 1 -- 172.21.15.6:0/2932987513 learned_addr learned my addr 172.21.15.6:0/2932987513 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.549+0000 7fc058ff9700 1 -- 172.21.15.6:0/1494883023 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fc04c0ce4a0 con 0x7fc05c06dfe0 2023-11-17T00:18:28.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 -- 172.21.15.6:0/2932987513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 msgr2=0x555b236a4f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236a4f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 -- 172.21.15.6:0/2932987513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 msgr2=0x555b236a9370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236a9370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 -- 172.21.15.6:0/2932987513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b23477630 con 0x555b236a98b0 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.550+0000 7fc05affd700 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc044009e20 tx=0x7fc044010650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65fb334c05de68d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.551+0000 7fc051ffb700 1 -- 172.21.15.6:0/2932987513 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044010e60 con 0x555b236a98b0 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.551+0000 7fc051ffb700 1 -- 172.21.15.6:0/2932987513 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc044004cf0 con 0x555b236a98b0 2023-11-17T00:18:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.551+0000 7fc051ffb700 1 -- 172.21.15.6:0/2932987513 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440126b0 con 0x555b236a98b0 2023-11-17T00:18:28.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.552+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2932987513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 msgr2=0x555b236a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.552+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc044009e20 tx=0x7fc044010650 comp rx=0 tx=0).stop 2023-11-17T00:18:28.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.552+0000 7fc05bfff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 0x7fc0480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.553+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2932987513 shutdown_connections 2023-11-17T00:18:28.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.553+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236a9d00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.553+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.553+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2932987513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.553+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2932987513 >> 172.21.15.6:0/2932987513 conn(0x555b2369a360 msgr2=0x555b2369fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.555+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2932987513 shutdown_connections 2023-11-17T00:18:28.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.555+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2932987513 wait complete. 2023-11-17T00:18:28.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.555+0000 7fc05bfff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 0x7fc0480a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc05c06a270 tx=0x7fc054009100 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.555+0000 7fc0757e0b80 1 -- start start 2023-11-17T00:18:28.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.556+0000 7fc0757e0b80 1 Processor -- start 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b235978e0 con 0x555b236a5470 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b23597580 con 0x555b236a98b0 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b23479920 con 0x555b236a4b60 2023-11-17T00:18:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.557+0000 7fc0757e0b80 10 client.0 _ll_register_callbacks cb 0x555b235b5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59758/0 (socket says 172.21.15.6:59758) 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 -- 172.21.15.6:0/2872745544 learned_addr learned my addr 172.21.15.6:0/2872745544 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 -- 172.21.15.6:0/2872745544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 msgr2=0x555b236b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 -- 172.21.15.6:0/2872745544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 msgr2=0x555b236ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.558+0000 7fc05bfff700 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.559+0000 7fc05bfff700 1 -- 172.21.15.6:0/2872745544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc044009a50 con 0x555b236a98b0 2023-11-17T00:18:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.559+0000 7fc05bfff700 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc0540093d0 tx=0x7fc054004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74dc4ca902ea63f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.560+0000 7fc03affd700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.560+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054015850 con 0x555b236a98b0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.560+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fc0540159b0 con 0x555b236a98b0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.560+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b23697b20 con 0x555b236a98b0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.561+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc054004910 con 0x555b236a98b0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.561+0000 7fc0757e0b80 10 client.27011 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.561+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555b235bc190 con 0x555b236a98b0 2023-11-17T00:18:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.561+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b23697b20 con 0x555b236a98b0 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.561+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc054021900 con 0x555b236a98b0 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 10 client.27011 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 10 client.27011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 20 client.27011 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b236be920 0x555b236c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 4 client.27011 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:18:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b235b80a0 con 0x555b236be920 2023-11-17T00:18:28.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.562+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b236be920 0x555b236c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.563+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b236be920 0x555b236c0d30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2782a3eb45a75c0 server_cookie=1fc5f4627b476e37 in_seq=0 out_seq=0 2023-11-17T00:18:28.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.563+0000 7fc03affd700 10 client.27011 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.563+0000 7fc03affd700 1 -- 172.21.15.6:0/2872745544 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b235b80a0 con 0x555b236be920 2023-11-17T00:18:28.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc03affd700 10 client.27011 handle_command_reply: tid=0 2023-11-17T00:18:28.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 client.27011 shutdown 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 msgr2=0x555b236aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc0540093d0 tx=0x7fc054004b40 comp rx=0 tx=0).stop 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 shutdown_connections 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236aea40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b236be920 0x555b236c0d30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/2872745544 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc03affd700 0 client.27011 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.564+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 >> 172.21.15.6:0/2872745544 conn(0x555b2369a360 msgr2=0x555b236aad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.565+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 shutdown_connections 2023-11-17T00:18:28.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.565+0000 7fc0757e0b80 1 -- 172.21.15.6:0/2872745544 wait complete. 2023-11-17T00:18:28.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.565+0000 7fc0757e0b80 20 client.27011 trim_cache size 0 max 16384 2023-11-17T00:18:28.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 Processor -- start 2023-11-17T00:18:28.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 -- start start 2023-11-17T00:18:28.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b2369f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b2369f4f0 con 0x555b236a5470 2023-11-17T00:18:28.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b236aed80 con 0x555b236a98b0 2023-11-17T00:18:28.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.602+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b236a8480 con 0x555b236a4b60 2023-11-17T00:18:28.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.603+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.603+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34534/0 (socket says 172.21.15.6:34534) 2023-11-17T00:18:28.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.603+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1723168327 learned_addr learned my addr 172.21.15.6:0/1723168327 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.603+0000 7fc05bfff700 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b2369f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1723168327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 msgr2=0x555b2369f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b2369f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1723168327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 msgr2=0x555b236b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 -- 172.21.15.6:0/1723168327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b235bbee0 con 0x555b236a4b60 2023-11-17T00:18:28.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.604+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c00 tx=0x7fc04c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22be49106a64f58f server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.605+0000 7fc0527fc700 1 -- 172.21.15.6:0/1723168327 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c09f180 con 0x555b236a4b60 2023-11-17T00:18:28.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.605+0000 7fc0527fc700 1 -- 172.21.15.6:0/1723168327 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04c09b320 con 0x555b236a4b60 2023-11-17T00:18:28.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.605+0000 7fc0527fc700 1 -- 172.21.15.6:0/1723168327 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c022850 con 0x555b236a4b60 2023-11-17T00:18:28.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 -- 172.21.15.6:0/1723168327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 msgr2=0x555b236b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c00 tx=0x7fc04c009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 -- 172.21.15.6:0/1723168327 shutdown_connections 2023-11-17T00:18:28.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b2369f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b59b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/1723168327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 -- 172.21.15.6:0/1723168327 >> 172.21.15.6:0/1723168327 conn(0x555b2369a360 msgr2=0x555b236a79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 -- 172.21.15.6:0/1723168327 shutdown_connections 2023-11-17T00:18:28.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.607+0000 7fc0757e0b80 1 -- 172.21.15.6:0/1723168327 wait complete. 2023-11-17T00:18:28.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.608+0000 7fc0757e0b80 1 -- start start 2023-11-17T00:18:28.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.608+0000 7fc0757e0b80 1 Processor -- start 2023-11-17T00:18:28.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b236b3fb0 con 0x555b236a5470 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b236b4120 con 0x555b236a98b0 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b236b42e0 con 0x555b236a4b60 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc0757e0b80 10 client.0 _ll_register_callbacks cb 0x7fc054004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:28.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc05b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:34540/0 (socket says 172.21.15.6:34540) 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc05b7fe700 1 -- 172.21.15.6:0/3548277375 learned_addr learned my addr 172.21.15.6:0/3548277375 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.609+0000 7fc05bfff700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.610+0000 7fc05b7fe700 1 -- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 msgr2=0x555b236b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.610+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.610+0000 7fc05b7fe700 1 -- 172.21.15.6:0/3548277375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 msgr2=0x555b236b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:28.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.610+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.610+0000 7fc05b7fe700 1 -- 172.21.15.6:0/3548277375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c11ac50 con 0x555b236a4b60 2023-11-17T00:18:28.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.611+0000 7fc03affd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:18:28.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.611+0000 7fc05b7fe700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc04c09b2b0 tx=0x7fc04c09b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3291c141343c2de server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.611+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c09f350 con 0x555b236a4b60 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.611+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fc04c0996c0 con 0x555b236a4b60 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc044004cf0 con 0x555b236a4b60 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc04c022850 con 0x555b236a4b60 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc0757e0b80 10 client.26983 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:28.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555b2340cc60 con 0x555b236a4b60 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04c0c9080 con 0x555b236a4b60 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.612+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc04c0ce070 con 0x555b236a4b60 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 10 client.26983 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 10 client.26983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 20 client.26983 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:28.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b2371bda0 0x555b2371e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 4 client.26983 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b235c4660 con 0x555b2371bda0 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.613+0000 7fc05affd700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b2371bda0 0x555b2371e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.614+0000 7fc05affd700 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b2371bda0 0x555b2371e1b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d477df904210fbb3 server_cookie=19c5bb1fa1e9d357 in_seq=0 out_seq=0 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.614+0000 7fc03affd700 10 client.26983 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.615+0000 7fc03affd700 1 -- 172.21.15.6:0/3548277375 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555b235c4660 con 0x555b2371bda0 2023-11-17T00:18:28.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.615+0000 7fc03affd700 10 client.26983 handle_command_reply: tid=0 2023-11-17T00:18:28.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.615+0000 7fc0757e0b80 1 client.26983 shutdown 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 msgr2=0x555b236b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc04c09b2b0 tx=0x7fc04c09b490 comp rx=0 tx=0).stop 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 shutdown_connections 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555b236a98b0 0x555b236b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555b236a4b60 0x555b236b52a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555b2371bda0 0x555b2371e1b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 --2- 172.21.15.6:0/3548277375 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555b236a5470 0x555b236b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc03affd700 0 client.26983 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.617+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 >> 172.21.15.6:0/3548277375 conn(0x555b2369a360 msgr2=0x555b236abc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.618+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 shutdown_connections 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.618+0000 7fc0757e0b80 1 -- 172.21.15.6:0/3548277375 wait complete. 2023-11-17T00:18:28.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.618+0000 7fc0757e0b80 20 client.26983 trim_cache size 0 max 16384 2023-11-17T00:18:28.624 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:18:28.624 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-17T00:18:28.624 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-17T00:18:28.624 INFO:teuthology.orchestra.run.smithi006.stdout: "17edca36-76c0-4085-8753-3691a0a6e07f": { 2023-11-17T00:18:28.624 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-17T00:18:28.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "17edca36-76c0-4085-8753-3691a0a6e07f", 2023-11-17T00:18:28.625 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive,force" 2023-11-17T00:18:28.625 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:18:28.625 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:18:28.625 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 msgr2=0x7fc0480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 0x7fc0480a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc05c06a270 tx=0x7fc054009100 comp rx=0 tx=0).stop 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 msgr2=0x7fc05c1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc04c005910 tx=0x7fc04c005740 comp rx=0 tx=0).stop 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 shutdown_connections 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc0480a1340 0x7fc0480a37f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc05c115260 0x7fc05c1a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c06dfe0 0x7fc05c1a0da0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 --2- 172.21.15.6:0/1494883023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc05c108030 0x7fc05c1a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.619+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 >> 172.21.15.6:0/1494883023 conn(0x7fc05c06c9d0 msgr2=0x7fc05c115630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.620+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 shutdown_connections 2023-11-17T00:18:28.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:28.620+0000 7fc061dff700 1 -- 172.21.15.6:0/1494883023 wait complete. 2023-11-17T00:18:28.654 INFO:tasks.cephfs.filesystem:scrub status for tag:17edca36-76c0-4085-8753-3691a0a6e07f - {'path': '/', 'tag': '17edca36-76c0-4085-8753-3691a0a6e07f', 'options': 'recursive,force'} 2023-11-17T00:18:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:28 smithi006 ceph-mon[97840]: pgmap v297: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:28 smithi006 ceph-mon[97840]: scrub queued for path: / 2023-11-17T00:18:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:28 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:28 smithi006 ceph-mon[97840]: scrub summary: active paths [/] 2023-11-17T00:18:28.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:28 smithi006 ceph-mon[97840]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:28 smithi087 ceph-mon[131182]: pgmap v297: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:28 smithi087 ceph-mon[131182]: scrub queued for path: / 2023-11-17T00:18:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:28 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:29.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:28 smithi087 ceph-mon[131182]: scrub summary: active paths [/] 2023-11-17T00:18:29.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:28 smithi087 ceph-mon[131182]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:28 smithi186 ceph-mon[131487]: pgmap v297: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:28 smithi186 ceph-mon[131487]: scrub queued for path: / 2023-11-17T00:18:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:28 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:28 smithi186 ceph-mon[131487]: scrub summary: active paths [/] 2023-11-17T00:18:29.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:28 smithi186 ceph-mon[131487]: scrub summary: idle+waiting paths [/] 2023-11-17T00:18:31.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:30 smithi087 ceph-mon[131182]: pgmap v298: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:31.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:30 smithi186 ceph-mon[131487]: pgmap v298: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:31.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:30 smithi006 ceph-mon[97840]: pgmap v298: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:32.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:31 smithi087 ceph-mon[131182]: scrub summary: idle 2023-11-17T00:18:32.121 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:31 smithi186 ceph-mon[131487]: scrub summary: idle 2023-11-17T00:18:32.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:31 smithi006 ceph-mon[97840]: scrub summary: idle 2023-11-17T00:18:33.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:32 smithi087 ceph-mon[131182]: pgmap v299: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:33.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:32 smithi186 ceph-mon[131487]: pgmap v299: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:33.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:32 smithi006 ceph-mon[97840]: pgmap v299: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:35.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:34 smithi087 ceph-mon[131182]: pgmap v300: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:35.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:34 smithi186 ceph-mon[131487]: pgmap v300: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:35.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:34 smithi006 ceph-mon[97840]: pgmap v300: 129 pgs: 70 active+clean+snaptrim_wait, 37 active+clean, 22 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:37.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:36 smithi087 ceph-mon[131182]: pgmap v301: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:37.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:36 smithi186 ceph-mon[131487]: pgmap v301: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:37.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:36 smithi006 ceph-mon[97840]: pgmap v301: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:39.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:38 smithi087 ceph-mon[131182]: pgmap v302: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:39.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:38 smithi186 ceph-mon[131487]: pgmap v302: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:39.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:38 smithi006 ceph-mon[97840]: pgmap v302: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:41.105 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:40 smithi087 ceph-mon[131182]: pgmap v303: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:41.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:40 smithi186 ceph-mon[131487]: pgmap v303: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:41.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:40 smithi006 ceph-mon[97840]: pgmap v303: 129 pgs: 63 active+clean+snaptrim_wait, 45 active+clean, 21 active+clean+snaptrim; 4.9 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s 2023-11-17T00:18:43.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:42 smithi087 ceph-mon[131182]: pgmap v304: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:43.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:42 smithi186 ceph-mon[131487]: pgmap v304: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:42 smithi006 ceph-mon[97840]: pgmap v304: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 3 op/s 2023-11-17T00:18:45.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:44 smithi087 ceph-mon[131182]: pgmap v305: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:45.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:44 smithi186 ceph-mon[131487]: pgmap v305: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:44 smithi006 ceph-mon[97840]: pgmap v305: 129 pgs: 55 active+clean+snaptrim_wait, 58 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:47.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:46 smithi087 ceph-mon[131182]: pgmap v306: 129 pgs: 55 active+clean+snaptrim_wait, 59 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:47.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:46 smithi186 ceph-mon[131487]: pgmap v306: 129 pgs: 55 active+clean+snaptrim_wait, 59 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:47.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:46 smithi006 ceph-mon[97840]: pgmap v306: 129 pgs: 55 active+clean+snaptrim_wait, 59 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:48 smithi087 ceph-mon[131182]: pgmap v307: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:48 smithi186 ceph-mon[131487]: pgmap v307: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:49.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:48 smithi006 ceph-mon[97840]: pgmap v307: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:51.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:50 smithi087 ceph-mon[131182]: pgmap v308: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:51.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:50 smithi186 ceph-mon[131487]: pgmap v308: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:50 smithi006 ceph-mon[97840]: pgmap v308: 129 pgs: 54 active+clean+snaptrim_wait, 59 active+clean, 16 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:52 smithi087 ceph-mon[131182]: pgmap v309: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:53.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:52 smithi186 ceph-mon[131487]: pgmap v309: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:52 smithi006 ceph-mon[97840]: pgmap v309: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:54 smithi087 ceph-mon[131182]: pgmap v310: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:55.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:54 smithi186 ceph-mon[131487]: pgmap v310: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:55.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:54 smithi006 ceph-mon[97840]: pgmap v310: 129 pgs: 46 active+clean+snaptrim_wait, 68 active+clean, 15 active+clean+snaptrim; 4.9 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2023-11-17T00:18:56.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-17T00:18:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 -- 172.21.15.6:0/2997173240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc107af0 msgr2=0x7fdcbc107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/2997173240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc107af0 0x7fdcbc107ec0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b0a0 tx=0x7fdcb400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 -- 172.21.15.6:0/2997173240 shutdown_connections 2023-11-17T00:18:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/2997173240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/2997173240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc107af0 0x7fdcbc107ec0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/2997173240 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc108400 0x7fdcbc112e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 -- 172.21.15.6:0/2997173240 >> 172.21.15.6:0/2997173240 conn(0x7fdcbc06c490 msgr2=0x7fdcbc071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 -- 172.21.15.6:0/2997173240 shutdown_connections 2023-11-17T00:18:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.954+0000 7fdcc21ff700 1 -- 172.21.15.6:0/2997173240 wait complete. 2023-11-17T00:18:56.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.956+0000 7fdcc21ff700 1 Processor -- start 2023-11-17T00:18:56.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.956+0000 7fdcc21ff700 1 -- start start 2023-11-17T00:18:56.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.956+0000 7fdcc21ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 0x7fdcbc10aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:56.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.956+0000 7fdcc21ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 0x7fdcbc10b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:56.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcc21ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc10b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcc21ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1b1540 con 0x7fdcbc1133b0 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcc21ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1a0c10 con 0x7fdcbc108400 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcc21ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1a0dd0 con 0x7fdcbc107af0 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcbaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 0x7fdcbc10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcbaffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 0x7fdcbc10b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:32928/0 (socket says 172.21.15.6:32928) 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcbaffd700 1 -- 172.21.15.6:0/275905336 learned_addr learned my addr 172.21.15.6:0/275905336 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:56.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcbb7fe700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 0x7fdcbc10aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:56.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.957+0000 7fdcbbfff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc10b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 -- 172.21.15.6:0/275905336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 msgr2=0x7fdcbc10b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 0x7fdcbc10b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 -- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 msgr2=0x7fdcbc10b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc10b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 -- 172.21.15.6:0/275905336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009d00 con 0x7fdcbc107af0 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbbfff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:56.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.958+0000 7fdcbb7fe700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 0x7fdcbc10aee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b070 tx=0x7fdcb400bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.959+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400e040 con 0x7fdcbc107af0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.959+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb4007930 con 0x7fdcbc107af0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.959+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb401b6c0 con 0x7fdcbc107af0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.959+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc118b30 con 0x7fdcbc107af0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.959+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc1189a0 con 0x7fdcbc107af0 2023-11-17T00:18:56.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.961+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc118cc0 con 0x7fdcbc107af0 2023-11-17T00:18:56.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.965+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdcb401d470 con 0x7fdcbc107af0 2023-11-17T00:18:56.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.967+0000 7fdcb8ff9700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 0x7fdca40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:56.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.968+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fdcb40ce0e0 con 0x7fdcbc107af0 2023-11-17T00:18:56.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.968+0000 7fdcbaffd700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 0x7fdca40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:56.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.969+0000 7fdcbaffd700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 0x7fdca40a3980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc10c6d0 tx=0x7fdcb0009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:56.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:56.970+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdcb40971b0 con 0x7fdcbc107af0 2023-11-17T00:18:56.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:56 smithi006 ceph-mon[97840]: pgmap v311: 129 pgs: 34 active+clean+snaptrim_wait, 83 active+clean, 12 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:56 smithi087 ceph-mon[131182]: pgmap v311: 129 pgs: 34 active+clean+snaptrim_wait, 83 active+clean, 12 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:57.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:56 smithi186 ceph-mon[131487]: pgmap v311: 129 pgs: 34 active+clean+snaptrim_wait, 83 active+clean, 12 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:18:57.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.409+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcbc10a320 con 0x7fdca40a14d0 2023-11-17T00:18:57.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.431+0000 7fdcb8ff9700 1 -- 172.21.15.6:0/275905336 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7fdcbc10a320 con 0x7fdca40a14d0 2023-11-17T00:18:57.434 INFO:teuthology.orchestra.run.smithi006.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-17T00:00:00", "created": "2023-11-17T00:08:54", "first": "2023-11-17T00:09:00", "last": "2023-11-17T00:18:00", "last_pruned": "2023-11-17T00:18:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-11-17T00:18:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 msgr2=0x7fdca40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 0x7fdca40a3980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc10c6d0 tx=0x7fdcb0009260 comp rx=0 tx=0).stop 2023-11-17T00:18:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 msgr2=0x7fdcbc10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 0x7fdcbc10aee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b070 tx=0x7fdcb400bd90 comp rx=0 tx=0).stop 2023-11-17T00:18:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 shutdown_connections 2023-11-17T00:18:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdca40a14d0 0x7fdca40a3980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdcbc1133b0 0x7fdcbc10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcbc107af0 0x7fdcbc10aee0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 --2- 172.21.15.6:0/275905336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcbc108400 0x7fdcbc10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.436+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 >> 172.21.15.6:0/275905336 conn(0x7fdcbc06c490 msgr2=0x7fdcbc108c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.437+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 shutdown_connections 2023-11-17T00:18:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.437+0000 7fdcc21ff700 1 -- 172.21.15.6:0/275905336 wait complete. 2023-11-17T00:18:57.470 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-17T00:18:57.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 -- 172.21.15.6:0/2480921507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 msgr2=0x7fee68068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:57.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/2480921507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68068740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee64009990 tx=0x7fee64009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 -- 172.21.15.6:0/2480921507 shutdown_connections 2023-11-17T00:18:57.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/2480921507 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68105330 0x7fee6810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/2480921507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68068740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/2480921507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68068d10 0x7fee68104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.806+0000 7fee6ef9d700 1 -- 172.21.15.6:0/2480921507 >> 172.21.15.6:0/2480921507 conn(0x7fee68074d00 msgr2=0x7fee68075100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:57.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.807+0000 7fee6ef9d700 1 -- 172.21.15.6:0/2480921507 shutdown_connections 2023-11-17T00:18:57.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.807+0000 7fee6ef9d700 1 -- 172.21.15.6:0/2480921507 wait complete. 2023-11-17T00:18:57.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.809+0000 7fee6ef9d700 1 Processor -- start 2023-11-17T00:18:57.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 -- start start 2023-11-17T00:18:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:57.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68068d10 0x7fee68109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68105330 0x7fee681a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:57.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681a28f0 con 0x7fee68068d10 2023-11-17T00:18:57.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681a2a60 con 0x7fee68068370 2023-11-17T00:18:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.812+0000 7fee6ef9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681a2c20 con 0x7fee68105330 2023-11-17T00:18:57.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:57.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:32950/0 (socket says 172.21.15.6:32950) 2023-11-17T00:18:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 -- 172.21.15.6:0/795656475 learned_addr learned my addr 172.21.15.6:0/795656475 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:57.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6e79c700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68105330 0x7fee681a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6d79a700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68068d10 0x7fee68109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 -- 172.21.15.6:0/795656475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68105330 msgr2=0x7fee681a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68105330 0x7fee681a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 -- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68068d10 msgr2=0x7fee68109e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:57.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68068d10 0x7fee68109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:57.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.813+0000 7fee6df9b700 1 -- 172.21.15.6:0/795656475 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640095f0 con 0x7fee68068370 2023-11-17T00:18:57.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.814+0000 7fee6df9b700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee64005910 tx=0x7fee64004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:57.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.814+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6401d070 con 0x7fee68068370 2023-11-17T00:18:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.815+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee640054f0 con 0x7fee68068370 2023-11-17T00:18:57.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.815+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64004c10 con 0x7fee68068370 2023-11-17T00:18:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.815+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee68100700 con 0x7fee68068370 2023-11-17T00:18:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.815+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee68100570 con 0x7fee68068370 2023-11-17T00:18:57.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.817+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee68100890 con 0x7fee68068370 2023-11-17T00:18:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.820+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fee6400fbc0 con 0x7fee68068370 2023-11-17T00:18:57.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.822+0000 7fee5effd700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 0x7fee540a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.823+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fee6409b210 con 0x7fee68068370 2023-11-17T00:18:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.823+0000 7fee6d79a700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 0x7fee540a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.824+0000 7fee6d79a700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 0x7fee540a3900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee6810af30 tx=0x7fee58009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:57.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:57.829+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee64097ce0 con 0x7fee68068370 2023-11-17T00:18:58.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.267+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 --> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fee6810ad10 con 0x7fee540a1450 2023-11-17T00:18:58.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.288+0000 7fee5effd700 1 -- 172.21.15.6:0/795656475 <== mgr.25469 v2:172.21.15.6:6800/3813072549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fee6810ad10 con 0x7fee540a1450 2023-11-17T00:18:58.291 INFO:teuthology.orchestra.run.smithi006.stdout:/ 1M 6M3h 2023-11-17T00:18:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 msgr2=0x7fee540a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 0x7fee540a3900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee6810af30 tx=0x7fee58009260 comp rx=0 tx=0).stop 2023-11-17T00:18:58.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 msgr2=0x7fee68109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:58.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee64005910 tx=0x7fee64004730 comp rx=0 tx=0).stop 2023-11-17T00:18:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 shutdown_connections 2023-11-17T00:18:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fee540a1450 0x7fee540a3900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee68105330 0x7fee681a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee68068370 0x7fee68109920 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 --2- 172.21.15.6:0/795656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fee68068d10 0x7fee68109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.292+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 >> 172.21.15.6:0/795656475 conn(0x7fee68074d00 msgr2=0x7fee680fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.293+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 shutdown_connections 2023-11-17T00:18:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:58.293+0000 7fee6ef9d700 1 -- 172.21.15.6:0/795656475 wait complete. 2023-11-17T00:18:58.326 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-11-17T00:18:58.339 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-11-17T00:18:58.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-17T00:18:59.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.004+0000 7fdc503ff700 1 -- 172.21.15.6:0/1651534739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48067dc0 msgr2=0x7fdc48068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.004+0000 7fdc503ff700 1 --2- 172.21.15.6:0/1651534739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48067dc0 0x7fdc48068230 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009990 tx=0x7fdc44009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.005+0000 7fdc503ff700 1 -- 172.21.15.6:0/1651534739 shutdown_connections 2023-11-17T00:18:59.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.005+0000 7fdc503ff700 1 --2- 172.21.15.6:0/1651534739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc48105250 0x7fdc48105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.005+0000 7fdc503ff700 1 --2- 172.21.15.6:0/1651534739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48068770 0x7fdc4810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.005+0000 7fdc503ff700 1 --2- 172.21.15.6:0/1651534739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48067dc0 0x7fdc48068230 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.005+0000 7fdc503ff700 1 -- 172.21.15.6:0/1651534739 >> 172.21.15.6:0/1651534739 conn(0x7fdc480763a0 msgr2=0x7fdc480767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.006+0000 7fdc503ff700 1 -- 172.21.15.6:0/1651534739 shutdown_connections 2023-11-17T00:18:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.006+0000 7fdc503ff700 1 -- 172.21.15.6:0/1651534739 wait complete. 2023-11-17T00:18:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.008+0000 7fdc503ff700 1 Processor -- start 2023-11-17T00:18:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.008+0000 7fdc503ff700 1 -- start start 2023-11-17T00:18:59.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc48067dc0 0x7fdc48196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 0x7fdc481969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 0x7fdc4819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819b440 con 0x7fdc48067dc0 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819b5b0 con 0x7fdc48105250 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc503ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819b770 con 0x7fdc48068770 2023-11-17T00:18:59.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.009+0000 7fdc4d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 0x7fdc481969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 0x7fdc481969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43962/0 (socket says 172.21.15.6:43962) 2023-11-17T00:18:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4d99a700 1 -- 172.21.15.6:0/2816573298 learned_addr learned my addr 172.21.15.6:0/2816573298 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 0x7fdc4819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2816573298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 msgr2=0x7fdc481969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 0x7fdc481969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc48067dc0 msgr2=0x7fdc48196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc48067dc0 0x7fdc48196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.010+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2816573298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440095f0 con 0x7fdc48105250 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 0x7fdc4819ae00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000ef90 tx=0x7fdc4000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc3f7fe700 1 -- 172.21.15.6:0/2816573298 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40018070 con 0x7fdc48105250 2023-11-17T00:18:59.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc48110420 con 0x7fdc48105250 2023-11-17T00:18:59.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc3f7fe700 1 -- 172.21.15.6:0/2816573298 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc48105250 2023-11-17T00:18:59.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc3f7fe700 1 -- 172.21.15.6:0/2816573298 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000cce0 con 0x7fdc48105250 2023-11-17T00:18:59.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.011+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc48110290 con 0x7fdc48105250 2023-11-17T00:18:59.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.017+0000 7fdc3f7fe700 1 -- 172.21.15.6:0/2816573298 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdc40003a00 con 0x7fdc48105250 2023-11-17T00:18:59.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.021+0000 7fdc63de8b80 1 Processor -- start 2023-11-17T00:18:59.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.021+0000 7fdc3f7fe700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 0x7fdc340a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.022+0000 7fdc3f7fe700 1 -- 172.21.15.6:0/2816573298 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fdc400ccd50 con 0x7fdc48105250 2023-11-17T00:18:59.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.022+0000 7fdc63de8b80 1 -- start start 2023-11-17T00:18:59.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee33c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 0x5578ee33cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee23d060 con 0x5578ee338470 2023-11-17T00:18:59.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee28f8a0 con 0x5578ee337b60 2023-11-17T00:18:59.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee288d30 con 0x5578ee33c8b0 2023-11-17T00:18:59.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 0x5578ee33cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 0x5578ee33cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43970/0 (socket says 172.21.15.6:43970) 2023-11-17T00:18:59.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4e19b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33948/0 (socket says 172.21.15.6:33948) 2023-11-17T00:18:59.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4e99c700 1 -- 172.21.15.6:0/1263700765 learned_addr learned my addr 172.21.15.6:0/1263700765 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.023+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 0x7fdc340a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 -- 172.21.15.6:0/1263700765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 msgr2=0x5578ee33cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 0x5578ee33cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 -- 172.21.15.6:0/1263700765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 msgr2=0x5578ee33c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-17T00:18:59.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee33c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 -- 172.21.15.6:0/1263700765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ee10a630 con 0x5578ee337b60 2023-11-17T00:18:59.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.024+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009bd0 tx=0x7fdc38009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c83764803b3834 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.025+0000 7fdc3cbfe700 1 -- 172.21.15.6:0/1263700765 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004230 con 0x5578ee337b60 2023-11-17T00:18:59.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.025+0000 7fdc3cbfe700 1 -- 172.21.15.6:0/1263700765 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3800edc0 con 0x5578ee337b60 2023-11-17T00:18:59.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.025+0000 7fdc3cbfe700 1 -- 172.21.15.6:0/1263700765 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38005020 con 0x5578ee337b60 2023-11-17T00:18:59.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 0x7fdc340a3870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b6f0 tx=0x7fdc4400b260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 -- 172.21.15.6:0/1263700765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 msgr2=0x5578ee337f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009bd0 tx=0x7fdc38009ee0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 -- 172.21.15.6:0/1263700765 shutdown_connections 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee33c8b0 0x5578ee33cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee337f30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/1263700765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee33c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.027+0000 7fdc63de8b80 1 -- 172.21.15.6:0/1263700765 >> 172.21.15.6:0/1263700765 conn(0x5578ee32d360 msgr2=0x5578ee332b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.028+0000 7fdc63de8b80 1 -- 172.21.15.6:0/1263700765 shutdown_connections 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.028+0000 7fdc63de8b80 1 -- 172.21.15.6:0/1263700765 wait complete. 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.029+0000 7fdc63de8b80 1 -- start start 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.030+0000 7fdc63de8b80 1 Processor -- start 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.031+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee341330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.031+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.031+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 0x5578ee341db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee22a8e0 con 0x5578ee33c8b0 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee22a580 con 0x5578ee337b60 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee10c920 con 0x5578ee338470 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc63de8b80 10 client.0 _ll_register_callbacks cb 0x5578ee248780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:43980/0 (socket says 172.21.15.6:43980) 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 -- 172.21.15.6:0/3347327385 learned_addr learned my addr 172.21.15.6:0/3347327385 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee341330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 0x5578ee341db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 -- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 msgr2=0x5578ee341330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee341330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 -- 172.21.15.6:0/3347327385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 msgr2=0x5578ee341db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 0x5578ee341db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4d99a700 1 -- 172.21.15.6:0/3347327385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc38009880 con 0x5578ee338470 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.032+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 0x5578ee341db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.033+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005460 tx=0x7fdc44009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac641aef0ee565c server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4402b070 con 0x5578ee338470 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fdc44004be0 con 0x5578ee338470 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578ee32ab20 con 0x5578ee338470 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc44004da0 con 0x5578ee338470 2023-11-17T00:18:59.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc63de8b80 10 client.27004 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.034+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5578ee24f190 con 0x5578ee338470 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578ee32ab20 con 0x5578ee338470 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc44024530 con 0x5578ee338470 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 10 client.27004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 20 client.27004 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee35c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 4 client.27004 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:18:59.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.035+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578ee24b0a0 con 0x5578ee35a070 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.036+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee35c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.037+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee35c480 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5daced36919c18c server_cookie=3a0a3c6283dc9ace in_seq=0 out_seq=0 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.037+0000 7fdc31ffb700 10 client.27004 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.038+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3347327385 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5578ee24b0a0 con 0x5578ee35a070 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.038+0000 7fdc31ffb700 10 client.27004 handle_command_reply: tid=0 2023-11-17T00:18:59.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.038+0000 7fdc63de8b80 1 client.27004 shutdown 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 msgr2=0x5578ee341870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005460 tx=0x7fdc44009ca0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 shutdown_connections 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee35c480 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc31ffb700 0 client.27004 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee33c8b0 0x5578ee341db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee337b60 0x5578ee341330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3347327385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee341870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 >> 172.21.15.6:0/3347327385 conn(0x5578ee32d360 msgr2=0x5578ee33dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 shutdown_connections 2023-11-17T00:18:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.039+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3347327385 wait complete. 2023-11-17T00:18:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.040+0000 7fdc63de8b80 20 client.27004 trim_cache size 0 max 16384 2023-11-17T00:18:59.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.070+0000 7fdc63de8b80 1 Processor -- start 2023-11-17T00:18:59.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.071+0000 7fdc63de8b80 1 -- start start 2023-11-17T00:18:59.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee337b60 0x5578ee33a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee3511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee333810 con 0x5578ee337b60 2023-11-17T00:18:59.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee33b520 con 0x5578ee33c8b0 2023-11-17T00:18:59.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.072+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee356ab0 con 0x5578ee338470 2023-11-17T00:18:59.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.073+0000 7fdc4d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee3511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.073+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.073+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33960/0 (socket says 172.21.15.6:33960) 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.073+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2604386023 learned_addr learned my addr 172.21.15.6:0/2604386023 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2604386023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 msgr2=0x5578ee3511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee3511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2604386023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee337b60 msgr2=0x5578ee33a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee337b60 0x5578ee33a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4e99c700 1 -- 172.21.15.6:0/2604386023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ee24eee0 con 0x5578ee33c8b0 2023-11-17T00:18:59.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.074+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee3511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.075+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc400c9fe0 tx=0x7fdc40099920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8fff00611d1d266 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.075+0000 7fdc3d3ff700 1 -- 172.21.15.6:0/2604386023 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40099a50 con 0x5578ee33c8b0 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.075+0000 7fdc3d3ff700 1 -- 172.21.15.6:0/2604386023 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4009da80 con 0x5578ee33c8b0 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.075+0000 7fdc3d3ff700 1 -- 172.21.15.6:0/2604386023 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40099f10 con 0x5578ee33c8b0 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 -- 172.21.15.6:0/2604386023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 msgr2=0x5578ee341be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc400c9fe0 tx=0x7fdc40099920 comp rx=0 tx=0).stop 2023-11-17T00:18:59.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 -- 172.21.15.6:0/2604386023 shutdown_connections 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee337b60 0x5578ee33a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee341be0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/2604386023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee338470 0x5578ee3511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.076+0000 7fdc63de8b80 1 -- 172.21.15.6:0/2604386023 >> 172.21.15.6:0/2604386023 conn(0x5578ee32d360 msgr2=0x5578ee33aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.077+0000 7fdc63de8b80 1 -- 172.21.15.6:0/2604386023 shutdown_connections 2023-11-17T00:18:59.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.077+0000 7fdc63de8b80 1 -- 172.21.15.6:0/2604386023 wait complete. 2023-11-17T00:18:59.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.077+0000 7fdc63de8b80 1 -- start start 2023-11-17T00:18:59.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.078+0000 7fdc63de8b80 1 Processor -- start 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.078+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 0x5578ee3a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.078+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee3a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc63de8b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee3a6840 con 0x5578ee338470 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee3a69b0 con 0x5578ee33c8b0 2023-11-17T00:18:59.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc63de8b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ee3a6b70 con 0x5578ee337b60 2023-11-17T00:18:59.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc63de8b80 10 client.0 _ll_register_callbacks cb 0x7fdc44005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:18:59.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc4e99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33962/0 (socket says 172.21.15.6:33962) 2023-11-17T00:18:59.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc4e99c700 1 -- 172.21.15.6:0/3688298172 learned_addr learned my addr 172.21.15.6:0/3688298172 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.079+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 0x5578ee3a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.080+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee3a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.080+0000 7fdc4e99c700 1 -- 172.21.15.6:0/3688298172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 msgr2=0x5578ee3a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.081+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 0x5578ee3a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.081+0000 7fdc4e99c700 1 -- 172.21.15.6:0/3688298172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 msgr2=0x5578ee3a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:18:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.081+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee3a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.081+0000 7fdc4e99c700 1 -- 172.21.15.6:0/3688298172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578ee24f190 con 0x5578ee33c8b0 2023-11-17T00:18:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.081+0000 7fdc4d99a700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee3a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 0x5578ee3a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc4e99c700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc400999e0 tx=0x7fdc40099690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa1a6a42b021f16 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40099eb0 con 0x5578ee33c8b0 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fdc40099a50 con 0x5578ee33c8b0 2023-11-17T00:18:59.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.082+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc3800edc0 con 0x5578ee33c8b0 2023-11-17T00:18:59.262 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:58 smithi186 ceph-mon[131487]: from='client.26986 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:18:59.262 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:58 smithi186 ceph-mon[131487]: pgmap v312: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.083+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc40099eb0 con 0x5578ee33c8b0 2023-11-17T00:18:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.083+0000 7fdc63de8b80 10 client.27047 fetch_fsmap learned FSMap version 22 2023-11-17T00:18:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.083+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fdc48110290 con 0x5578ee33c8b0 2023-11-17T00:18:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.084+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3800edc0 con 0x5578ee33c8b0 2023-11-17T00:18:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.084+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc40098ac0 con 0x5578ee33c8b0 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.085+0000 7fdc63de8b80 10 client.27047 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.085+0000 7fdc63de8b80 10 client.27047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.085+0000 7fdc63de8b80 20 client.27047 populate_metadata read hostname 'smithi006' 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.085+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee34b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.086+0000 7fdc63de8b80 4 client.27047 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.086+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578ee257660 con 0x5578ee35a070 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.086+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee34b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.088+0000 7fdc4e19b700 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee34b3f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e808fe63ace0413 server_cookie=fa3b809d480798d7 in_seq=0 out_seq=0 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.088+0000 7fdc31ffb700 10 client.27047 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc31ffb700 1 -- 172.21.15.6:0/3688298172 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5578ee257660 con 0x5578ee35a070 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc31ffb700 10 client.27047 handle_command_reply: tid=0 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 client.27047 shutdown 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 msgr2=0x5578ee3a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc400999e0 tx=0x7fdc40099690 comp rx=0 tx=0).stop 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 shutdown_connections 2023-11-17T00:18:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578ee33c8b0 0x5578ee3a6300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5578ee337b60 0x5578ee3a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x5578ee35a070 0x5578ee34b3f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 --2- 172.21.15.6:0/3688298172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5578ee338470 0x5578ee3a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 >> 172.21.15.6:0/3688298172 conn(0x5578ee32d360 msgr2=0x5578ee3418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.089+0000 7fdc31ffb700 0 client.27047 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.090+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 shutdown_connections 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.090+0000 7fdc63de8b80 1 -- 172.21.15.6:0/3688298172 wait complete. 2023-11-17T00:18:59.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.090+0000 7fdc63de8b80 20 client.27047 trim_cache size 0 max 16384 2023-11-17T00:18:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:18:59.268 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-17T00:18:59.268 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-17T00:18:59.268 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 msgr2=0x7fdc340a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 0x7fdc340a3870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b6f0 tx=0x7fdc4400b260 comp rx=0 tx=0).stop 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 msgr2=0x7fdc4819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 0x7fdc4819ae00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000ef90 tx=0x7fdc4000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 shutdown_connections 2023-11-17T00:18:59.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc340a13c0 0x7fdc340a3870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc48067dc0 0x7fdc48196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc48105250 0x7fdc4819ae00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 --2- 172.21.15.6:0/2816573298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc48068770 0x7fdc481969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 >> 172.21.15.6:0/2816573298 conn(0x7fdc480763a0 msgr2=0x7fdc480fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.092+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 shutdown_connections 2023-11-17T00:18:59.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.093+0000 7fdc503ff700 1 -- 172.21.15.6:0/2816573298 wait complete. 2023-11-17T00:18:59.271 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:58 smithi006 ceph-mon[97840]: from='client.26986 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:18:59.271 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:58 smithi006 ceph-mon[97840]: pgmap v312: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:59.272 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-17T00:18:59.272 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:18:59.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:58 smithi087 ceph-mon[131182]: from='client.26986 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:18:59.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:58 smithi087 ceph-mon[131182]: pgmap v312: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:18:59.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.582+0000 7fdc33dff700 1 -- 172.21.15.6:0/1743736318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c105620 msgr2=0x7fdc2c10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.582+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1743736318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c105620 0x7fdc2c10d1a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b0a0 tx=0x7fdc2400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 -- 172.21.15.6:0/1743736318 shutdown_connections 2023-11-17T00:18:59.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1743736318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1743736318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c105620 0x7fdc2c10d1a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1743736318 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c101660 0x7fdc2c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 -- 172.21.15.6:0/1743736318 >> 172.21.15.6:0/1743736318 conn(0x7fdc2c0774d0 msgr2=0x7fdc2c0778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:18:59.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.584+0000 7fdc33dff700 1 -- 172.21.15.6:0/1743736318 shutdown_connections 2023-11-17T00:18:59.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.585+0000 7fdc33dff700 1 -- 172.21.15.6:0/1743736318 wait complete. 2023-11-17T00:18:59.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.586+0000 7fdc33dff700 1 Processor -- start 2023-11-17T00:18:59.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.586+0000 7fdc33dff700 1 -- start start 2023-11-17T00:18:59.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc2c101660 0x7fdc2c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c105620 0x7fdc2c073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc31b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a8f40 con 0x7fdc2c101660 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a9080 con 0x7fdc2c105620 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc33dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1a91c0 con 0x7fdc2c100cc0 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc31b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44022/0 (socket says 172.21.15.6:44022) 2023-11-17T00:18:59.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.587+0000 7fdc31b9b700 1 -- 172.21.15.6:0/1781375696 learned_addr learned my addr 172.21.15.6:0/1781375696 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:18:59.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc3239c700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c105620 0x7fdc2c073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 -- 172.21.15.6:0/1781375696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c105620 msgr2=0x7fdc2c073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c105620 0x7fdc2c073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 -- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc2c101660 msgr2=0x7fdc2c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc2c101660 0x7fdc2c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 -- 172.21.15.6:0/1781375696 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009d00 con 0x7fdc2c100cc0 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.588+0000 7fdc31b9b700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800d950 tx=0x7fdc2800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.589+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28009940 con 0x7fdc2c100cc0 2023-11-17T00:18:59.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.589+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc28010460 con 0x7fdc2c100cc0 2023-11-17T00:18:59.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.589+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800f620 con 0x7fdc2c100cc0 2023-11-17T00:18:59.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.589+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c1103e0 con 0x7fdc2c100cc0 2023-11-17T00:18:59.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.589+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c110250 con 0x7fdc2c100cc0 2023-11-17T00:18:59.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.591+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c110570 con 0x7fdc2c100cc0 2023-11-17T00:18:59.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.594+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdc280108d0 con 0x7fdc2c100cc0 2023-11-17T00:18:59.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.595+0000 7fdc22ffd700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 0x7fdc180a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:18:59.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.595+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fdc280ccf80 con 0x7fdc2c100cc0 2023-11-17T00:18:59.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.596+0000 7fdc3139a700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 0x7fdc180a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:18:59.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.597+0000 7fdc3139a700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 0x7fdc180a3870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0745e0 tx=0x7fdc1c009620 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:18:59.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:18:59.600+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdc28095f30 con 0x7fdc2c100cc0 2023-11-17T00:19:00.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.017+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc2c067af0 con 0x7fdc2c100cc0 2023-11-17T00:19:00.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.018+0000 7fdc22ffd700 1 -- 172.21.15.6:0/1781375696 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7fdc28016090 con 0x7fdc2c100cc0 2023-11-17T00:19:00.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:19:00.021 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:19:00.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.021+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 msgr2=0x7fdc180a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.021+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 0x7fdc180a3870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0745e0 tx=0x7fdc1c009620 comp rx=0 tx=0).stop 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.021+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 msgr2=0x7fdc2c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.021+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800d950 tx=0x7fdc2800dc60 comp rx=0 tx=0).stop 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 shutdown_connections 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fdc180a13c0 0x7fdc180a3870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c105620 0x7fdc2c073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc2c100cc0 0x7fdc2c072df0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 --2- 172.21.15.6:0/1781375696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc2c101660 0x7fdc2c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 >> 172.21.15.6:0/1781375696 conn(0x7fdc2c0774d0 msgr2=0x7fdc2c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 shutdown_connections 2023-11-17T00:19:00.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.022+0000 7fdc33dff700 1 -- 172.21.15.6:0/1781375696 wait complete. 2023-11-17T00:19:00.026 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:19:00.057 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-17T00:19:00.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:18:59 smithi006 ceph-mon[97840]: from='client.27029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:19:00.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:18:59 smithi087 ceph-mon[131182]: from='client.27029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:19:00.370 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:18:59 smithi186 ceph-mon[131487]: from='client.27029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-17T00:19:00.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 -- 172.21.15.6:0/4235759380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 msgr2=0x7fa0601135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 --2- 172.21.15.6:0/4235759380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa0601135a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b0a0 tx=0x7fa05800b3b0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 -- 172.21.15.6:0/4235759380 shutdown_connections 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 --2- 172.21.15.6:0/4235759380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075020 0x7fa0600753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 --2- 172.21.15.6:0/4235759380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa0601135a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 --2- 172.21.15.6:0/4235759380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075930 0x7fa060110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 -- 172.21.15.6:0/4235759380 >> 172.21.15.6:0/4235759380 conn(0x7fa0600fdfe0 msgr2=0x7fa0601003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.378+0000 7fa0669ff700 1 -- 172.21.15.6:0/4235759380 shutdown_connections 2023-11-17T00:19:00.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.379+0000 7fa0669ff700 1 -- 172.21.15.6:0/4235759380 wait complete. 2023-11-17T00:19:00.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.380+0000 7fa0669ff700 1 Processor -- start 2023-11-17T00:19:00.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.380+0000 7fa0669ff700 1 -- start start 2023-11-17T00:19:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.381+0000 7fa0669ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.381+0000 7fa0669ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075930 0x7fa0601a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa0669ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa06019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa0669ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019cf90 con 0x7fa060075930 2023-11-17T00:19:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:33994/0 (socket says 172.21.15.6:33994) 2023-11-17T00:19:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa0669ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019d100 con 0x7fa060075020 2023-11-17T00:19:00.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 -- 172.21.15.6:0/3718269902 learned_addr learned my addr 172.21.15.6:0/3718269902 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:00.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06019d2c0 con 0x7fa0601111c0 2023-11-17T00:19:00.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075930 0x7fa0601a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa06019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 -- 172.21.15.6:0/3718269902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 msgr2=0x7fa06019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa06019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 -- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075930 msgr2=0x7fa0601a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075930 0x7fa0601a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.382+0000 7fa05ffff700 1 -- 172.21.15.6:0/3718269902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058009d00 con 0x7fa060075020 2023-11-17T00:19:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b530 tx=0x7fa05000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa05d7fa700 1 -- 172.21.15.6:0/3718269902 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050011830 con 0x7fa060075020 2023-11-17T00:19:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa05d7fa700 1 -- 172.21.15.6:0/3718269902 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa050011e50 con 0x7fa060075020 2023-11-17T00:19:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa05d7fa700 1 -- 172.21.15.6:0/3718269902 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000f2e0 con 0x7fa060075020 2023-11-17T00:19:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa060116a10 con 0x7fa060075020 2023-11-17T00:19:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.383+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060116880 con 0x7fa060075020 2023-11-17T00:19:00.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.388+0000 7fa05d7fa700 1 -- 172.21.15.6:0/3718269902 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa050010380 con 0x7fa060075020 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.390+0000 7fa07a263b80 1 Processor -- start 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 -- start start 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa05d7fa700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 0x7fa0480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa05d7fa700 1 -- 172.21.15.6:0/3718269902 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa0500ccb10 con 0x7fa060075020 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636e8b0 0x555c5636ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5626f060 con 0x555c5636e8b0 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c562c18a0 con 0x555c56369b60 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.391+0000 7fa07a263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c562bad30 con 0x555c5636a470 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 0x7fa0480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34010/0 (socket says 172.21.15.6:34010) 2023-11-17T00:19:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 -- 172.21.15.6:0/790774137 learned_addr learned my addr 172.21.15.6:0/790774137 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:00.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 -- 172.21.15.6:0/790774137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 msgr2=0x555c5636e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 -- 172.21.15.6:0/790774137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636e8b0 msgr2=0x555c5636ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636e8b0 0x555c5636ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.392+0000 7fa05ffff700 1 -- 172.21.15.6:0/790774137 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5613c630 con 0x555c56369b60 2023-11-17T00:19:00.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.393+0000 7fa05ffff700 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa050099880 tx=0x7fa0500998f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65dbeaf2fb0bc14 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.393+0000 7fa0467fc700 1 -- 172.21.15.6:0/790774137 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050010660 con 0x555c56369b60 2023-11-17T00:19:00.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.393+0000 7fa0467fc700 1 -- 172.21.15.6:0/790774137 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0500107c0 con 0x555c56369b60 2023-11-17T00:19:00.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.393+0000 7fa0467fc700 1 -- 172.21.15.6:0/790774137 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500a18b0 con 0x555c56369b60 2023-11-17T00:19:00.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 -- 172.21.15.6:0/790774137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 msgr2=0x555c56369f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 0x7fa0480a37f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa058009cd0 tx=0x7fa058009b80 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa050099880 tx=0x7fa0500998f0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 -- 172.21.15.6:0/790774137 shutdown_connections 2023-11-17T00:19:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636e8b0 0x555c5636ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56369f30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 --2- 172.21.15.6:0/790774137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 -- 172.21.15.6:0/790774137 >> 172.21.15.6:0/790774137 conn(0x555c5635f360 msgr2=0x555c56364b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 -- 172.21.15.6:0/790774137 shutdown_connections 2023-11-17T00:19:00.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.395+0000 7fa07a263b80 1 -- 172.21.15.6:0/790774137 wait complete. 2023-11-17T00:19:00.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.396+0000 7fa07a263b80 1 -- start start 2023-11-17T00:19:00.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.396+0000 7fa07a263b80 1 Processor -- start 2023-11-17T00:19:00.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56376050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636a470 0x555c56373340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56373a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5625c8e0 con 0x555c5636a470 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5625c580 con 0x555c56369b60 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5613e920 con 0x555c5636e8b0 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.397+0000 7fa07a263b80 10 client.0 _ll_register_callbacks cb 0x555c5627a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa064f9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56373a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa064f9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56373a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44058/0 (socket says 172.21.15.6:44058) 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa064f9c700 1 -- 172.21.15.6:0/3851206420 learned_addr learned my addr 172.21.15.6:0/3851206420 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:00.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56376050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636a470 0x555c56373340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.398+0000 7fa05ffff700 1 -- 172.21.15.6:0/3851206420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 msgr2=0x555c56373a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56373a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa05ffff700 1 -- 172.21.15.6:0/3851206420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636a470 msgr2=0x555c56373340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636a470 0x555c56373340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa05ffff700 1 -- 172.21.15.6:0/3851206420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060116880 con 0x555c56369b60 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56376050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa050011040 tx=0x7fa050099ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e067a72feac43e server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.399+0000 7fa03f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050010660 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mon.2 v2:172.21.15.87:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa050098ff0 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5635cb20 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0500a18b0 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa07a263b80 10 client.27062 fetch_fsmap learned FSMap version 22 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c56281190 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.400+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mon.2 v2:172.21.15.87:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa05007c080 con 0x555c56369b60 2023-11-17T00:19:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mon.2 v2:172.21.15.87:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0500b90d0 con 0x555c56369b60 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 10 client.27062 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 20 client.27062 populate_metadata read hostname 'smithi006' 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c56383490 0x555c563858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 4 client.27062 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c5627d0a0 con 0x555c56383490 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.401+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c56383490 0x555c563858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.402+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c56383490 0x555c563858a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4a6f5e848561bd server_cookie=a876419bb5c44370 in_seq=0 out_seq=0 2023-11-17T00:19:00.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.402+0000 7fa03f7fe700 10 client.27062 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.403+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3851206420 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555c5627d0a0 con 0x555c56383490 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.403+0000 7fa03f7fe700 10 client.27062 handle_command_reply: tid=0 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.403+0000 7fa07a263b80 1 client.27062 shutdown 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.403+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 msgr2=0x555c56376050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.403+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56376050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa050011040 tx=0x7fa050099ef0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 shutdown_connections 2023-11-17T00:19:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56373a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c56369b60 0x555c56376050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c56383490 0x555c563858a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3851206420 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c5636a470 0x555c56373340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 >> 172.21.15.6:0/3851206420 conn(0x555c5635f360 msgr2=0x555c5636fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa03f7fe700 0 client.27062 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 shutdown_connections 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.404+0000 7fa07a263b80 1 -- 172.21.15.6:0/3851206420 wait complete. 2023-11-17T00:19:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.405+0000 7fa07a263b80 20 client.27062 trim_cache size 0 max 16384 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.427+0000 7fa07a263b80 1 Processor -- start 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 -- start start 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c5637a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5635ef10 con 0x555c56369b60 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c56380330 con 0x555c5636e8b0 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.428+0000 7fa07a263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c563661c0 con 0x555c5636a470 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34032/0 (socket says 172.21.15.6:34032) 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 -- 172.21.15.6:0/1099558690 learned_addr learned my addr 172.21.15.6:0/1099558690 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa05f7fe700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 -- 172.21.15.6:0/1099558690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 msgr2=0x555c5636bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa05ffff700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c5637a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 -- 172.21.15.6:0/1099558690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 msgr2=0x555c5637a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c5637a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa064f9c700 1 -- 172.21.15.6:0/1099558690 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c56360b00 con 0x555c5636e8b0 2023-11-17T00:19:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.429+0000 7fa05ffff700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c5637a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.430+0000 7fa064f9c700 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b070 tx=0x7fa05801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6515eb386fb597c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.431+0000 7fa046ffd700 1 -- 172.21.15.6:0/1099558690 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800e050 con 0x555c5636e8b0 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.431+0000 7fa046ffd700 1 -- 172.21.15.6:0/1099558690 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa058007820 con 0x555c5636e8b0 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.431+0000 7fa046ffd700 1 -- 172.21.15.6:0/1099558690 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05801c670 con 0x555c5636e8b0 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 -- 172.21.15.6:0/1099558690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 msgr2=0x555c56375bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b070 tx=0x7fa05801d000 comp rx=0 tx=0).stop 2023-11-17T00:19:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 -- 172.21.15.6:0/1099558690 shutdown_connections 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c5637a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636e8b0 0x555c56375bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 --2- 172.21.15.6:0/1099558690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636a470 0x555c5636bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 -- 172.21.15.6:0/1099558690 >> 172.21.15.6:0/1099558690 conn(0x555c5635f360 msgr2=0x555c5637fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 -- 172.21.15.6:0/1099558690 shutdown_connections 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.432+0000 7fa07a263b80 1 -- 172.21.15.6:0/1099558690 wait complete. 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.433+0000 7fa07a263b80 1 -- start start 2023-11-17T00:19:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.433+0000 7fa07a263b80 1 Processor -- start 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c56363550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 0x555c56363a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56363fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c56364510 con 0x555c56369b60 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c56364680 con 0x555c5636a470 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c56364840 con 0x555c5636e8b0 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa07a263b80 10 client.0 _ll_register_callbacks cb 0x7fa05400a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-17T00:19:00.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa05f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 0x555c56363a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa05f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 0x555c56363a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34040/0 (socket says 172.21.15.6:34040) 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa05f7fe700 1 -- 172.21.15.6:0/3642490324 learned_addr learned my addr 172.21.15.6:0/3642490324 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.434+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56363fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c56363550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa064f9c700 1 -- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 msgr2=0x555c56363a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 0x555c56363a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa064f9c700 1 -- 172.21.15.6:0/3642490324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 msgr2=0x555c56363550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c56363550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa064f9c700 1 -- 172.21.15.6:0/3642490324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060113fd0 con 0x555c5636e8b0 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.435+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c56363550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.436+0000 7fa064f9c700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56363fd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b0a0 tx=0x7fa058005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f439541ff25793 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.436+0000 7fa03f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-11-17T00:19:00.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.436+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800e040 con 0x555c5636e8b0 2023-11-17T00:19:00.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.436+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa058007990 con 0x555c5636e8b0 2023-11-17T00:19:00.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.436+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0500107c0 con 0x555c5636e8b0 2023-11-17T00:19:00.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa058031410 con 0x555c5636e8b0 2023-11-17T00:19:00.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 10 client.27034 fetch_fsmap learned FSMap version 22 2023-11-17T00:19:00.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c560e8de0 con 0x555c5636e8b0 2023-11-17T00:19:00.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa058043030 con 0x555c5636e8b0 2023-11-17T00:19:00.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 22) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa05801c680 con 0x555c5636e8b0 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 10 client.27034 fetch_fsmap finished waiting for FSMap version 22 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 20 client.27034 populate_metadata read hostname 'smithi006' 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c563d7c10 0x555c563da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 4 client.27034 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-11-17T00:19:00.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.437+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 --> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c56289660 con 0x555c563d7c10 2023-11-17T00:19:00.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.438+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c563d7c10 0x555c563da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa05ffff700 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c563d7c10 0x555c563da020 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c7a70dae337c79 server_cookie=38a3b7f8a6785660 in_seq=0 out_seq=0 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa03f7fe700 10 client.27034 ms_handle_connect on v2:172.21.15.87:6838/804288298 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa03f7fe700 1 -- 172.21.15.6:0/3642490324 <== mds.0 v2:172.21.15.87:6838/804288298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c56289660 con 0x555c563d7c10 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa03f7fe700 10 client.27034 handle_command_reply: tid=0 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa07a263b80 1 client.27034 shutdown 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 msgr2=0x555c56363fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.439+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56363fd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b0a0 tx=0x7fa058005540 comp rx=0 tx=0).stop 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 shutdown_connections 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:6838/804288298,v1:172.21.15.87:6839/804288298] conn(0x555c563d7c10 0x555c563da020 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x555c56369b60 0x555c56363550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555c5636e8b0 0x555c56363fd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 --2- 172.21.15.6:0/3642490324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x555c5636a470 0x555c56363a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 >> 172.21.15.6:0/3642490324 conn(0x555c5635f360 msgr2=0x555c5636d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa03f7fe700 0 client.27034 ms_handle_reset on v2:172.21.15.87:6838/804288298 2023-11-17T00:19:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 shutdown_connections 2023-11-17T00:19:00.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 1 -- 172.21.15.6:0/3642490324 wait complete. 2023-11-17T00:19:00.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.440+0000 7fa07a263b80 20 client.27034 trim_cache size 0 max 16384 2023-11-17T00:19:00.454 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 msgr2=0x7fa0480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 0x7fa0480a37f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa058009cd0 tx=0x7fa058009b80 comp rx=0 tx=0).stop 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 msgr2=0x7fa0601a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b530 tx=0x7fa05000b840 comp rx=0 tx=0).stop 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 shutdown_connections 2023-11-17T00:19:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fa0480a1340 0x7fa0480a37f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0601111c0 0x7fa06019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa060075020 0x7fa0601a28b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 --2- 172.21.15.6:0/3718269902 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa060075930 0x7fa0601a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.441+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 >> 172.21.15.6:0/3718269902 conn(0x7fa0600fdfe0 msgr2=0x7fa0601003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.442+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 shutdown_connections 2023-11-17T00:19:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:00.442+0000 7fa0669ff700 1 -- 172.21.15.6:0/3718269902 wait complete. 2023-11-17T00:19:01.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:00 smithi006 ceph-mon[97840]: pgmap v313: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:01.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:00 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1781375696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:19:01.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:00 smithi087 ceph-mon[131182]: pgmap v313: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:01.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:00 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1781375696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:19:01.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:00 smithi186 ceph-mon[131487]: pgmap v313: 129 pgs: 12 active+clean+snaptrim_wait, 109 active+clean, 8 active+clean+snaptrim; 4.9 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:01.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:00 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1781375696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:19:01.475 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-11-17T00:19:01.475 INFO:tasks.fwd_scrub:done joining 2023-11-17T00:19:01.475 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-17T00:19:01.488 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:19:01.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.719+0000 7f78561ff700 1 -- 172.21.15.6:0/2046257927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850108560 msgr2=0x7f7850108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:01.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.719+0000 7f78561ff700 1 --2- 172.21.15.6:0/2046257927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850108560 0x7f7850108930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7840009990 tx=0x7f7840009ca0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 -- 172.21.15.6:0/2046257927 shutdown_connections 2023-11-17T00:19:01.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 --2- 172.21.15.6:0/2046257927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850102ee0 0x7f785010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 --2- 172.21.15.6:0/2046257927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850108560 0x7f7850108930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 --2- 172.21.15.6:0/2046257927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102530 0x7f78501029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 -- 172.21.15.6:0/2046257927 >> 172.21.15.6:0/2046257927 conn(0x7f78500fdff0 msgr2=0x7f7850100400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:01.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 -- 172.21.15.6:0/2046257927 shutdown_connections 2023-11-17T00:19:01.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.720+0000 7f78561ff700 1 -- 172.21.15.6:0/2046257927 wait complete. 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 Processor -- start 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 -- start start 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102ee0 0x7f7850198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 0x7f785019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:01.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019d560 con 0x7f7850108560 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019d6d0 con 0x7f7850102ee0 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.721+0000 7f78561ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019d890 con 0x7f7850102530 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44094/0 (socket says 172.21.15.6:44094) 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 -- 172.21.15.6:0/2057632540 learned_addr learned my addr 172.21.15.6:0/2057632540 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784effd700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102ee0 0x7f7850198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784ffff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 0x7f785019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:01.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 -- 172.21.15.6:0/2057632540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102ee0 msgr2=0x7f7850198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102ee0 0x7f7850198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 -- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 msgr2=0x7f785019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 0x7f785019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 -- 172.21.15.6:0/2057632540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400095f0 con 0x7f7850102530 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784ffff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 0x7f785019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.722+0000 7f784f7fe700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7840005910 tx=0x7f7840005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.723+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784001d070 con 0x7f7850102530 2023-11-17T00:19:01.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.723+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7850112580 con 0x7f7850102530 2023-11-17T00:19:01.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.723+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7840022470 con 0x7f7850102530 2023-11-17T00:19:01.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.723+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840017650 con 0x7f7850102530 2023-11-17T00:19:01.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.723+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78501123f0 con 0x7f7850102530 2023-11-17T00:19:01.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.724+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850112710 con 0x7f7850102530 2023-11-17T00:19:01.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.726+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f78400225e0 con 0x7f7850102530 2023-11-17T00:19:01.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.728+0000 7f784cff9700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 0x7f78380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:01.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.728+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f78400ce690 con 0x7f7850102530 2023-11-17T00:19:01.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.728+0000 7f784effd700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 0x7f78380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:01.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.728+0000 7f784effd700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 0x7f78380a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7850199b90 tx=0x7f7844009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:01.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.731+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78400976d0 con 0x7f7850102530 2023-11-17T00:19:01.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.978+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7850199970 con 0x7f7850102530 2023-11-17T00:19:01.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.980+0000 7f784cff9700 1 -- 172.21.15.6:0/2057632540 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+25168 (secure 0 0 0) 0x7f784009acf0 con 0x7f7850102530 2023-11-17T00:19:01.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:19:01.982 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":100,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:18:06.606656+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6800/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1767647821":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/2273227531":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/4272143857":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3662483346":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:19:01.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 msgr2=0x7f78380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:01.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 0x7f78380a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7850199b90 tx=0x7f7844009260 comp rx=0 tx=0).stop 2023-11-17T00:19:01.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 msgr2=0x7f7850198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:01.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7840005910 tx=0x7f7840005720 comp rx=0 tx=0).stop 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 shutdown_connections 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f78380a12c0 0x7f78380a3770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7850108560 0x7f785019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7850102530 0x7f7850198580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 --2- 172.21.15.6:0/2057632540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7850102ee0 0x7f7850198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 >> 172.21.15.6:0/2057632540 conn(0x7f78500fdff0 msgr2=0x7f78501092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 shutdown_connections 2023-11-17T00:19:01.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:01.983+0000 7f78561ff700 1 -- 172.21.15.6:0/2057632540 wait complete. 2023-11-17T00:19:02.007 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:19:02.007 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:19:02.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/3348119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf4100570 msgr2=0x7f6cf4100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/3348119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf4100570 0x7f6cf4100940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009990 tx=0x7f6cf0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/3348119141 shutdown_connections 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/3348119141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf41055b0 0x7f6cf410d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/3348119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf4100570 0x7f6cf4100940 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/3348119141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100f10 0x7f6cf4104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/3348119141 >> 172.21.15.6:0/3348119141 conn(0x7f6cf40770e0 msgr2=0x7f6cf40774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/3348119141 shutdown_connections 2023-11-17T00:19:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.232+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/3348119141 wait complete. 2023-11-17T00:19:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.233+0000 7f6cfb9ff700 1 Processor -- start 2023-11-17T00:19:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.233+0000 7f6cfb9ff700 1 -- start start 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.233+0000 7f6cfb9ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 0x7f6cf4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.233+0000 7f6cfb9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cfb9ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf41055b0 0x7f6cf41a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cfb9ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4073b60 con 0x7f6cf4100570 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cfb9ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf41a4b30 con 0x7f6cf41055b0 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cfb9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf41a4c70 con 0x7f6cf4100f10 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44112/0 (socket says 172.21.15.6:44112) 2023-11-17T00:19:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 -- 172.21.15.6:0/2561900165 learned_addr learned my addr 172.21.15.6:0/2561900165 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf9f9c700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf41055b0 0x7f6cf41a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf979b700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 0x7f6cf4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 -- 172.21.15.6:0/2561900165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf41055b0 msgr2=0x7f6cf41a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf41055b0 0x7f6cf41a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 -- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 msgr2=0x7f6cf4072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 0x7f6cf4072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 -- 172.21.15.6:0/2561900165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00095f0 con 0x7f6cf4100f10 2023-11-17T00:19:02.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf979b700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 0x7f6cf4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.234+0000 7f6cf8f9a700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400e940 tx=0x7f6ce400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.235+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4009730 con 0x7f6cf4100f10 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.235+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce400ee90 con 0x7f6cf4100f10 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.235+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf41103c0 con 0x7f6cf4100f10 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.235+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4010ea0 con 0x7f6cf4100f10 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.235+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf4110230 con 0x7f6cf4100f10 2023-11-17T00:19:02.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.236+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf4110550 con 0x7f6cf4100f10 2023-11-17T00:19:02.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.239+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6ce400cbd0 con 0x7f6cf4100f10 2023-11-17T00:19:02.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.240+0000 7f6cea7fc700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 0x7f6ce00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.240+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f6ce4014070 con 0x7f6cf4100f10 2023-11-17T00:19:02.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.241+0000 7f6cf979b700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 0x7f6ce00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.241+0000 7f6cf979b700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 0x7f6ce00a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf0005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:02.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.244+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ce4051730 con 0x7f6cf4100f10 2023-11-17T00:19:02.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.473+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cf41a5650 con 0x7f6cf4100f10 2023-11-17T00:19:02.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.474+0000 7f6cea7fc700 1 -- 172.21.15.6:0/2561900165 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ce4096340 con 0x7f6cf4100f10 2023-11-17T00:19:02.476 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:19:02.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 msgr2=0x7f6ce00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 0x7f6ce00a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf0005fb0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 msgr2=0x7f6cf4073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400e940 tx=0x7f6ce400ec50 comp rx=0 tx=0).stop 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 shutdown_connections 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ce00a12c0 0x7f6ce00a3770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6cf4100570 0x7f6cf4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cf41055b0 0x7f6cf41a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 --2- 172.21.15.6:0/2561900165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cf4100f10 0x7f6cf4073330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 >> 172.21.15.6:0/2561900165 conn(0x7f6cf40770e0 msgr2=0x7f6cf4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:02.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 shutdown_connections 2023-11-17T00:19:02.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.476+0000 7f6cfb9ff700 1 -- 172.21.15.6:0/2561900165 wait complete. 2023-11-17T00:19:02.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:19:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 -- 172.21.15.6:0/1500009961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac1015f0 msgr2=0x7f59ac104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 --2- 172.21.15.6:0/1500009961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac1015f0 0x7f59ac104e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59a00099c0 tx=0x7f59a0009cd0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 -- 172.21.15.6:0/1500009961 shutdown_connections 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 --2- 172.21.15.6:0/1500009961 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 0x7f59ac101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 --2- 172.21.15.6:0/1500009961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac105560 0x7f59ac10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 --2- 172.21.15.6:0/1500009961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac1015f0 0x7f59ac104e80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 -- 172.21.15.6:0/1500009961 >> 172.21.15.6:0/1500009961 conn(0x7f59ac0774f0 msgr2=0x7f59ac0778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 -- 172.21.15.6:0/1500009961 shutdown_connections 2023-11-17T00:19:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.727+0000 7f59b0bff700 1 -- 172.21.15.6:0/1500009961 wait complete. 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.728+0000 7f59b0bff700 1 Processor -- start 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.728+0000 7f59b0bff700 1 -- start start 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 0x7f59ac072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac1015f0 0x7f59ac073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac073b60 con 0x7f59ac100c50 2023-11-17T00:19:02.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac1a4aa0 con 0x7f59ac105560 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59b0bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac1a4be0 con 0x7f59ac1015f0 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59aad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59a9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac1015f0 0x7f59ac073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59aad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34100/0 (socket says 172.21.15.6:34100) 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59aad9d700 1 -- 172.21.15.6:0/689580638 learned_addr learned my addr 172.21.15.6:0/689580638 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.729+0000 7f59aa59c700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 0x7f59ac072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 -- 172.21.15.6:0/689580638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac1015f0 msgr2=0x7f59ac073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac1015f0 0x7f59ac073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 -- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 msgr2=0x7f59ac072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 0x7f59ac072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 -- 172.21.15.6:0/689580638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00095f0 con 0x7f59ac105560 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59aad9d700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ef30 tx=0x7f59a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4018070 con 0x7f59ac105560 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59a400f040 con 0x7f59ac105560 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac110320 con 0x7f59ac105560 2023-11-17T00:19:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400cd00 con 0x7f59ac105560 2023-11-17T00:19:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.730+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac110190 con 0x7f59ac105560 2023-11-17T00:19:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.731+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac1104b0 con 0x7f59ac105560 2023-11-17T00:19:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.733+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f59a4003a00 con 0x7f59ac105560 2023-11-17T00:19:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.735+0000 7f599b7fe700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 0x7f59940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.735+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f59a40ccbf0 con 0x7f59ac105560 2023-11-17T00:19:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.735+0000 7f59aa59c700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 0x7f59940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:02.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.736+0000 7f59aa59c700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 0x7f59940a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f599c009c60 tx=0x7f599c009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:02.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.739+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f59a4095c30 con 0x7f59ac105560 2023-11-17T00:19:02.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.973+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59ac1a55b0 con 0x7f59ac105560 2023-11-17T00:19:02.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.974+0000 7f599b7fe700 1 -- 172.21.15.6:0/689580638 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59a4099250 con 0x7f59ac105560 2023-11-17T00:19:02.976 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:19:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.976+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 msgr2=0x7f59940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.976+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 0x7f59940a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f599c009c60 tx=0x7f599c009260 comp rx=0 tx=0).stop 2023-11-17T00:19:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 msgr2=0x7f59ac1a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ef30 tx=0x7f59a400c3a0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 shutdown_connections 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f59940a12c0 0x7f59940a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59ac100c50 0x7f59ac072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ac105560 0x7f59ac1a4560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 --2- 172.21.15.6:0/689580638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59ac1015f0 0x7f59ac073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 >> 172.21.15.6:0/689580638 conn(0x7f59ac0774f0 msgr2=0x7f59ac0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 shutdown_connections 2023-11-17T00:19:02.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:02.977+0000 7f59b0bff700 1 -- 172.21.15.6:0/689580638 wait complete. 2023-11-17T00:19:02.999 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:19:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:02 smithi006 ceph-mon[97840]: pgmap v314: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:02 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2057632540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:19:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:02 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2561900165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:19:03.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.221+0000 7fd3369ff700 1 -- 172.21.15.6:0/1861491811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3301015f0 msgr2=0x7fd330104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.221+0000 7fd3369ff700 1 --2- 172.21.15.6:0/1861491811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3301015f0 0x7fd330104e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd324009990 tx=0x7fd324009ca0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 -- 172.21.15.6:0/1861491811 shutdown_connections 2023-11-17T00:19:03.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 --2- 172.21.15.6:0/1861491811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd330100c50 0x7fd330101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 --2- 172.21.15.6:0/1861491811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330105560 0x7fd33010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 --2- 172.21.15.6:0/1861491811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3301015f0 0x7fd330104e80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 -- 172.21.15.6:0/1861491811 >> 172.21.15.6:0/1861491811 conn(0x7fd330077500 msgr2=0x7fd330077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:03.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 -- 172.21.15.6:0/1861491811 shutdown_connections 2023-11-17T00:19:03.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.222+0000 7fd3369ff700 1 -- 172.21.15.6:0/1861491811 wait complete. 2023-11-17T00:19:03.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 Processor -- start 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 -- start start 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd3301015f0 0x7fd330198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd330105560 0x7fd33019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33019d5d0 con 0x7fd3301015f0 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33019d740 con 0x7fd330100c50 2023-11-17T00:19:03.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.223+0000 7fd3369ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33019d900 con 0x7fd330105560 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34128/0 (socket says 172.21.15.6:34128) 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 -- 172.21.15.6:0/3894725740 learned_addr learned my addr 172.21.15.6:0/3894725740 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32f7fe700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd3301015f0 0x7fd330198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd334f9c700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd330105560 0x7fd33019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 -- 172.21.15.6:0/3894725740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd330105560 msgr2=0x7fd33019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd330105560 0x7fd33019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 -- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd3301015f0 msgr2=0x7fd330198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd3301015f0 0x7fd330198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 -- 172.21.15.6:0/3894725740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240095f0 con 0x7fd330100c50 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.224+0000 7fd32ffff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e9a0 tx=0x7fd32000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.225+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320009790 con 0x7fd330100c50 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.225+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd320010430 con 0x7fd330100c50 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.225+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330110320 con 0x7fd330100c50 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.225+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320018680 con 0x7fd330100c50 2023-11-17T00:19:03.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.225+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd330110190 con 0x7fd330100c50 2023-11-17T00:19:03.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.226+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3301104b0 con 0x7fd330100c50 2023-11-17T00:19:03.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.228+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd3200105a0 con 0x7fd330100c50 2023-11-17T00:19:03.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.230+0000 7fd32d7fa700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 0x7fd3180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.230+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd320014070 con 0x7fd330100c50 2023-11-17T00:19:03.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.230+0000 7fd32f7fe700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 0x7fd3180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.231+0000 7fd32f7fe700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 0x7fd3180a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd32400b2e0 tx=0x7fd324005fd0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:03.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.236+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd320051730 con 0x7fd330100c50 2023-11-17T00:19:03.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:02 smithi087 ceph-mon[131182]: pgmap v314: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:03.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:02 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2057632540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:19:03.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:02 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2561900165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:19:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:02 smithi186 ceph-mon[131487]: pgmap v314: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:02 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2057632540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:19:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:02 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2561900165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:19:03.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.468+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3301a5740 con 0x7fd330100c50 2023-11-17T00:19:03.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.468+0000 7fd32d7fa700 1 -- 172.21.15.6:0/3894725740 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3200962a0 con 0x7fd330100c50 2023-11-17T00:19:03.471 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:19:03.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 msgr2=0x7fd3180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 0x7fd3180a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd32400b2e0 tx=0x7fd324005fd0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 msgr2=0x7fd3301985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e9a0 tx=0x7fd32000ecb0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 shutdown_connections 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd3180a1340 0x7fd3180a37f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd330105560 0x7fd33019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd330100c50 0x7fd3301985f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 --2- 172.21.15.6:0/3894725740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd3301015f0 0x7fd330198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 >> 172.21.15.6:0/3894725740 conn(0x7fd330077500 msgr2=0x7fd3300feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:03.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 shutdown_connections 2023-11-17T00:19:03.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.471+0000 7fd3369ff700 1 -- 172.21.15.6:0/3894725740 wait complete. 2023-11-17T00:19:03.495 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:19:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 -- 172.21.15.6:0/2924089826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 msgr2=0x7f700c068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 --2- 172.21.15.6:0/2924089826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c068230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0099c0 tx=0x7f6ffc009cd0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 -- 172.21.15.6:0/2924089826 shutdown_connections 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 --2- 172.21.15.6:0/2924089826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c1052a0 0x7f700c105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 --2- 172.21.15.6:0/2924089826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c068770 0x7f700c10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 --2- 172.21.15.6:0/2924089826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c068230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 -- 172.21.15.6:0/2924089826 >> 172.21.15.6:0/2924089826 conn(0x7f700c0763a0 msgr2=0x7f700c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 -- 172.21.15.6:0/2924089826 shutdown_connections 2023-11-17T00:19:03.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.718+0000 7f7013dff700 1 -- 172.21.15.6:0/2924089826 wait complete. 2023-11-17T00:19:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.719+0000 7f7013dff700 1 Processor -- start 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 -- start start 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c068770 0x7f700c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c1052a0 0x7f700c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19b340 con 0x7f700c068770 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19b4b0 con 0x7f700c067dc0 2023-11-17T00:19:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.720+0000 7f7013dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19b670 con 0x7f700c1052a0 2023-11-17T00:19:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:34146/0 (socket says 172.21.15.6:34146) 2023-11-17T00:19:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 -- 172.21.15.6:0/3892768086 learned_addr learned my addr 172.21.15.6:0/3892768086 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f701139a700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c068770 0x7f700c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f701239c700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c1052a0 0x7f700c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 -- 172.21.15.6:0/3892768086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c1052a0 msgr2=0x7f700c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c1052a0 0x7f700c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 -- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c068770 msgr2=0x7f700c1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c068770 0x7f700c1968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 -- 172.21.15.6:0/3892768086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc0095f0 con 0x7f700c067dc0 2023-11-17T00:19:03.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.721+0000 7f7011b9b700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f700800e9a0 tx=0x7f700800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.722+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008009790 con 0x7f700c067dc0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.722+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7008010430 con 0x7f700c067dc0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.722+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008018680 con 0x7f700c067dc0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.722+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c110390 con 0x7f700c067dc0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.722+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c110200 con 0x7f700c067dc0 2023-11-17T00:19:03.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.723+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c110520 con 0x7f700c067dc0 2023-11-17T00:19:03.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.725+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f70080105a0 con 0x7f700c067dc0 2023-11-17T00:19:03.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.727+0000 7f7002ffd700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 0x7f6ff80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:03.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.727+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f7008014070 con 0x7f700c067dc0 2023-11-17T00:19:03.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.727+0000 7f701139a700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 0x7f6ff80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:03.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.728+0000 7f701139a700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 0x7f6ff80a37f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009990 tx=0x7f6ffc005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:03.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.728+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f70080962b0 con 0x7f700c067dc0 2023-11-17T00:19:03.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.988+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f700c1a33d0 con 0x7f700c067dc0 2023-11-17T00:19:03.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.988+0000 7f7002ffd700 1 -- 172.21.15.6:0/3892768086 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70080998d0 con 0x7f700c067dc0 2023-11-17T00:19:03.990 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:19:03.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 msgr2=0x7f6ff80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 0x7f6ff80a37f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009990 tx=0x7f6ffc005fb0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 msgr2=0x7f700c196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f700800e9a0 tx=0x7f700800ecb0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 shutdown_connections 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ff80a1340 0x7f6ff80a37f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f700c1052a0 0x7f700c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f700c067dc0 0x7f700c196360 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 --2- 172.21.15.6:0/3892768086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f700c068770 0x7f700c1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:03.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 >> 172.21.15.6:0/3892768086 conn(0x7f700c0763a0 msgr2=0x7f700c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:03.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 shutdown_connections 2023-11-17T00:19:03.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:03.991+0000 7f7013dff700 1 -- 172.21.15.6:0/3892768086 wait complete. 2023-11-17T00:19:04.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-17T00:19:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:03 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/689580638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:03 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3894725740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 -- 172.21.15.6:0/740968478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 msgr2=0x7f310c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:04.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 --2- 172.21.15.6:0/740968478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c10f310 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f310400b0a0 tx=0x7f310400b3b0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 -- 172.21.15.6:0/740968478 shutdown_connections 2023-11-17T00:19:04.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 --2- 172.21.15.6:0/740968478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c10f310 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 --2- 172.21.15.6:0/740968478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1084e0 0x7f310c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 --2- 172.21.15.6:0/740968478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1024d0 0x7f310c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.228+0000 7f3110fff700 1 -- 172.21.15.6:0/740968478 >> 172.21.15.6:0/740968478 conn(0x7f310c0fdf90 msgr2=0x7f310c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:04.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.229+0000 7f3110fff700 1 -- 172.21.15.6:0/740968478 shutdown_connections 2023-11-17T00:19:04.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.229+0000 7f3110fff700 1 -- 172.21.15.6:0/740968478 wait complete. 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 Processor -- start 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 -- start start 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 0x7f310c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 0x7f310c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:04.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19c5e0 con 0x7f310c102e80 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19c750 con 0x7f310c1084e0 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3110fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19c910 con 0x7f310c1024d0 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f3109d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f310a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 0x7f310c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f310a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 0x7f310c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:44184/0 (socket says 172.21.15.6:44184) 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f310a59c700 1 -- 172.21.15.6:0/3663365122 learned_addr learned my addr 172.21.15.6:0/3663365122 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:19:04.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.230+0000 7f310ad9d700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 0x7f310c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 -- 172.21.15.6:0/3663365122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 msgr2=0x7f310c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 0x7f310c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 -- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 msgr2=0x7f310c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 -- 172.21.15.6:0/3663365122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009d00 con 0x7f310c1084e0 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f3109d9b700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f310ad9d700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 0x7f310c19bfe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3104009fd0 tx=0x7f310400ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:04.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310400e040 con 0x7f310c1084e0 2023-11-17T00:19:04.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3104004e60 con 0x7f310c1084e0 2023-11-17T00:19:04.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c112520 con 0x7f310c1084e0 2023-11-17T00:19:04.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.231+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104005560 con 0x7f310c1084e0 2023-11-17T00:19:04.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.232+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c112390 con 0x7f310c1084e0 2023-11-17T00:19:04.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.233+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c19ce60 con 0x7f310c1084e0 2023-11-17T00:19:04.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.234+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f310401dbd0 con 0x7f310c1084e0 2023-11-17T00:19:04.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.236+0000 7f30fb7fe700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 0x7f30f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:19:04.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.236+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f31040ce9b0 con 0x7f310c1084e0 2023-11-17T00:19:04.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.236+0000 7f310a59c700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 0x7f30f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:19:04.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.237+0000 7f310a59c700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 0x7f30f40a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005950 tx=0x7f30fc00b500 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:19:04.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.240+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31040979f0 con 0x7f310c1084e0 2023-11-17T00:19:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:03 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/689580638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:03 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3894725740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:03 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/689580638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:03 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3894725740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:19:04.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.510+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f310c1126b0 con 0x7f310c1084e0 2023-11-17T00:19:04.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.519+0000 7f30fb7fe700 1 -- 172.21.15.6:0/3663365122 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733765 (secure 0 0 0) 0x7f310409b010 con 0x7f310c1084e0 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout: "epoch": 26, 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout: "active_gid": 25469, 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout: "active_name": "x", 2023-11-17T00:19:04.531 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addrs": { 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6800", 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3813072549 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:6801", 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3813072549 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "active_addr": "172.21.15.6:6801/3813072549", 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "active_change": "2023-11-17T00:08:53.335939+0000", 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "active_mgr_features": 4540138322906710015, 2023-11-17T00:19:04.533 INFO:teuthology.orchestra.run.smithi006.stdout: "available": true, 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "standbys": [ 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 25447, 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "z", 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540138322906710015, 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:04.535 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2023-11-17T00:19:04.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.537 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:04.538 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.539 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.540 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.561 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.561 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:04.563 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.564 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2023-11-17T00:19:04.566 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.567 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.568 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2023-11-17T00:19:04.569 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.570 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.572 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.572 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.572 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2023-11-17T00:19:04.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.626 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2023-11-17T00:19:04.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.630 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2023-11-17T00:19:04.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:04.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:04.645 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2023-11-17T00:19:04.646 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.647 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2023-11-17T00:19:04.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.649 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2023-11-17T00:19:04.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2023-11-17T00:19:04.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-17T00:19:04.653 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.654 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2023-11-17T00:19:04.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2023-11-17T00:19:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.657 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.658 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:04.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:04.661 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:04.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:04.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.662 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-17T00:19:04.662 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:04.662 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.663 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:04.664 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:04.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:04.669 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.670 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2023-11-17T00:19:04.673 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-17T00:19:04.674 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.675 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2023-11-17T00:19:04.676 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap" 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.677 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:04.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:04.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:04.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:04.899 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.899 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.900 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2023-11-17T00:19:04.902 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-17T00:19:04.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2023-11-17T00:19:04.904 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.905 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.933 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:04.933 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.935 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2023-11-17T00:19:04.936 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.937 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_password": { 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_password", 2023-11-17T00:19:04.938 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.939 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web password", 2023-11-17T00:19:04.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.940 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.940 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.940 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_user": { 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_user", 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.941 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web user", 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2023-11-17T00:19:04.942 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2023-11-17T00:19:04.943 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.953 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.954 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.955 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2023-11-17T00:19:04.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-17T00:19:04.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.958 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.958 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.976 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.976 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.978 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2023-11-17T00:19:04.980 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.981 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:04.982 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:04.983 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:04.984 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:04.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:04.984 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2023-11-17T00:19:05.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.034 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2023-11-17T00:19:05.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2023-11-17T00:19:05.036 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.037 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.038 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.039 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.040 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.041 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2023-11-17T00:19:05.042 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.043 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.044 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.045 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-17T00:19:05.046 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.047 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.048 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.049 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.050 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.051 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.051 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.053 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.053 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:05.053 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.053 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.054 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.055 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.056 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2023-11-17T00:19:05.056 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2023-11-17T00:19:05.057 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.057 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.057 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.057 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.058 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.059 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.059 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.059 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2023-11-17T00:19:05.117 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2023-11-17T00:19:05.117 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.118 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.118 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.119 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.119 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.119 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2023-11-17T00:19:05.119 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.120 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.121 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.122 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.134 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.135 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2023-11-17T00:19:05.136 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.137 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2023-11-17T00:19:05.139 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.141 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.141 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2023-11-17T00:19:05.141 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.141 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.143 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.144 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.145 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.146 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.147 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.148 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.149 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.150 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.151 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.152 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.153 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.154 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-17T00:19:05.155 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.156 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-17T00:19:05.157 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.158 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2023-11-17T00:19:05.158 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.158 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2023-11-17T00:19:05.159 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.160 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2023-11-17T00:19:05.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.162 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.163 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2023-11-17T00:19:05.164 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.165 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.166 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2023-11-17T00:19:05.167 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2023-11-17T00:19:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-17T00:19:05.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.170 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.170 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.170 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.170 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_password": { 2023-11-17T00:19:05.170 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_password", 2023-11-17T00:19:05.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.173 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:05.173 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.173 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.173 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.174 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web password", 2023-11-17T00:19:05.174 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.174 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.174 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_user": { 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_user", 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.175 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web user", 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2023-11-17T00:19:05.176 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.177 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-17T00:19:05.178 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.178 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.178 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.178 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.179 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.180 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-17T00:19:05.182 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.183 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.184 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.186 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.187 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.188 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.189 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2023-11-17T00:19:05.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2023-11-17T00:19:05.189 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.189 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2023-11-17T00:19:05.191 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.192 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2023-11-17T00:19:05.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-17T00:19:05.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.195 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.196 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.201 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.202 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.203 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.205 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.206 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.207 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.208 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2023-11-17T00:19:05.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.213 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-17T00:19:05.214 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2023-11-17T00:19:05.215 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.219 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.220 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2023-11-17T00:19:05.221 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.222 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.223 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.224 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.225 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.226 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.231 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.232 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.234 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-17T00:19:05.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-17T00:19:05.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.237 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.237 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.238 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.296 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.296 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.296 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.296 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.297 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.298 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.300 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.300 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.301 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-17T00:19:05.302 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.303 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.305 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.305 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:05.305 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.305 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-17T00:19:05.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.308 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.317 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.317 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.318 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.319 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-17T00:19:05.320 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.321 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.322 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.323 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.324 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-17T00:19:05.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.326 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.327 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.327 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.348 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.348 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2023-11-17T00:19:05.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.349 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.350 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.350 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.350 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.350 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-17T00:19:05.350 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-17T00:19:05.352 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.352 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.352 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.354 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-17T00:19:05.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.393 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.394 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.399 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.399 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.400 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.401 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.402 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-17T00:19:05.402 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-17T00:19:05.402 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.402 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.402 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.403 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.404 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-17T00:19:05.405 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.406 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.408 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-17T00:19:05.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.410 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.411 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.412 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.412 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.412 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.413 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.413 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.413 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.417 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.418 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2023-11-17T00:19:05.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-17T00:19:05.422 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.424 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.425 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.426 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-17T00:19:05.427 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.429 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2023-11-17T00:19:05.432 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.434 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.436 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.438 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.439 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.440 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.441 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.441 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.441 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.441 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.441 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.442 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.444 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.445 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.446 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.446 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.446 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.446 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.447 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.448 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2023-11-17T00:19:05.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.450 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.451 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.452 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.453 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:05.454 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.455 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.457 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:05.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.462 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.463 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.464 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.465 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.466 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.469 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.470 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.471 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.472 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.473 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.474 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2023-11-17T00:19:05.475 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2023-11-17T00:19:05.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.478 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.482 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2023-11-17T00:19:05.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-17T00:19:05.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2023-11-17T00:19:05.485 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.486 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.487 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.488 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.489 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.490 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.491 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.493 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.494 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.495 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.496 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:05.497 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2023-11-17T00:19:05.499 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.500 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2023-11-17T00:19:05.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-17T00:19:05.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.505 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.506 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:05.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-17T00:19:05.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.509 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.510 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.511 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.512 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.513 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.514 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.515 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.516 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.517 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.518 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2023-11-17T00:19:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:05.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.523 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.524 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2023-11-17T00:19:05.525 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.526 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2023-11-17T00:19:05.527 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.528 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.529 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.533 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.534 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.535 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.536 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.537 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.538 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.539 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.542 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.543 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.544 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2023-11-17T00:19:05.545 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2023-11-17T00:19:05.546 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.547 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.548 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.549 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.550 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.551 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.552 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.554 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.557 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.558 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-17T00:19:05.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.712 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.713 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.714 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.714 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.714 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.767 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.768 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.769 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.769 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2023-11-17T00:19:05.772 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2023-11-17T00:19:05.773 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.774 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2023-11-17T00:19:05.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.776 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.776 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2023-11-17T00:19:05.776 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.776 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.776 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.777 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2023-11-17T00:19:05.777 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2023-11-17T00:19:05.777 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.777 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.780 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.780 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.780 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2023-11-17T00:19:05.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.781 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.782 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2023-11-17T00:19:05.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-17T00:19:05.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2023-11-17T00:19:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.785 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.786 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.787 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.788 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.790 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.791 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.794 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.796 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.797 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.798 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.800 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.802 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.802 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.802 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.832 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.835 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.835 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.854 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.857 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.858 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.860 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.861 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.884 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.884 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.884 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.884 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.891 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.891 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.914 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.915 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.915 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.916 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.917 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.918 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.919 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.956 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.957 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.958 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.960 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.961 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.961 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.981 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.982 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.983 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2023-11-17T00:19:05.984 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.985 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.986 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.987 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.988 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.989 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.991 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.992 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.993 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:05.994 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:05.995 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:05.996 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.997 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:05.998 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:05.999 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.000 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.003 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.005 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.007 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.008 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.009 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.009 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.011 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.012 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.013 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "noautoscale": { 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "noautoscale", 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.014 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "global autoscale flag", 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2023-11-17T00:19:06.018 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2023-11-17T00:19:06.020 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.020 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.020 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.021 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.022 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.023 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.024 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.025 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.027 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.028 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.029 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.032 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2023-11-17T00:19:06.033 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.034 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.034 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.034 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2023-11-17T00:19:06.035 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.035 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.035 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.036 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:06.037 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.037 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.037 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.037 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2023-11-17T00:19:06.037 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.067 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.068 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.070 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.071 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.072 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.073 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-17T00:19:06.073 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-17T00:19:06.073 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.073 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.092 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.093 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.094 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.095 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.096 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.097 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.098 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.099 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.099 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.099 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.099 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.108 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.109 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.110 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.111 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.112 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.113 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.114 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.115 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.117 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.118 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.119 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:06.120 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:06.121 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.121 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.121 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:06.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.122 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.122 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:06.123 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.124 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.125 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-17T00:19:06.125 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.125 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.125 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.168 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.168 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2023-11-17T00:19:06.168 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2023-11-17T00:19:06.168 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.168 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.169 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.170 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.171 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.172 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:06.173 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:06.173 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:06.173 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.173 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.173 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.174 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.174 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.175 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.175 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.175 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.175 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2023-11-17T00:19:06.175 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.176 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.176 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.178 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.181 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.182 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.183 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.184 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.185 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.186 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.186 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.186 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.186 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2023-11-17T00:19:06.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.190 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.191 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.192 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.193 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.196 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.197 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.199 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.200 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.201 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.202 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.203 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.204 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.205 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.206 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.208 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.212 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.213 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.214 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.216 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.219 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.222 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2023-11-17T00:19:06.223 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2023-11-17T00:19:06.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-17T00:19:06.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.226 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.226 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.227 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.232 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.233 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.234 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2023-11-17T00:19:06.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-17T00:19:06.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.237 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.238 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.239 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.241 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.242 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.243 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.244 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.245 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2023-11-17T00:19:06.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.250 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2023-11-17T00:19:06.252 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.253 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.254 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.255 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.256 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2023-11-17T00:19:06.257 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.258 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.259 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.260 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.261 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.262 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.263 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.264 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.265 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.267 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.268 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.269 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.270 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2023-11-17T00:19:06.271 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.272 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.273 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.274 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.275 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.276 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.277 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.279 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.281 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.282 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.282 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.283 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.283 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.284 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.285 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.286 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.287 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.288 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.289 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.290 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.291 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.292 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.293 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.294 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2023-11-17T00:19:06.295 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.296 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.297 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.300 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.301 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.302 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.303 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.304 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.305 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2023-11-17T00:19:06.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.309 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.310 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.311 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.312 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.313 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.314 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.315 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.316 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.317 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.319 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.319 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.319 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.319 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2023-11-17T00:19:06.322 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.323 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.325 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.326 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.327 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.328 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.331 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.333 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.335 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.336 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-17T00:19:06.337 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.338 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.339 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.340 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2023-11-17T00:19:06.341 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2023-11-17T00:19:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.343 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2023-11-17T00:19:06.344 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.345 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.346 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.347 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.348 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.349 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.350 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.351 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.352 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.353 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.358 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.358 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.358 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.358 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2023-11-17T00:19:06.359 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.360 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2023-11-17T00:19:06.361 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.362 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.363 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.364 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.365 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.366 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.367 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.368 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.369 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.370 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.371 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.372 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.373 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.376 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.377 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.378 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.379 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.380 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.381 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.381 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.381 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.381 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.381 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.382 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.406 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.407 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2023-11-17T00:19:06.408 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.409 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.410 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.411 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.412 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.413 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2023-11-17T00:19:06.414 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.415 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.416 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.417 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.417 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.418 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.418 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.420 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.421 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.422 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.423 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.423 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.423 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.423 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.424 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.425 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.426 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.427 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.428 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.428 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.428 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.428 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.428 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.429 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2023-11-17T00:19:06.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.432 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.434 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.435 INFO:teuthology.orchestra.run.smithi006.stdout: "gid": 25463, 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "y", 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_features": 4540138322906710015, 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.436 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.437 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.438 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.439 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.441 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.443 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.444 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.446 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2023-11-17T00:19:06.447 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2023-11-17T00:19:06.448 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.449 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.450 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.451 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2023-11-17T00:19:06.451 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2023-11-17T00:19:06.455 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.456 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.457 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.458 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.459 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2023-11-17T00:19:06.460 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2023-11-17T00:19:06.461 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.462 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.464 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.465 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2023-11-17T00:19:06.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.469 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:06.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2023-11-17T00:19:06.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-17T00:19:06.472 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.473 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.474 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2023-11-17T00:19:06.475 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2023-11-17T00:19:06.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:06.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:06.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.481 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.482 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.484 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.485 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.486 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.489 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.491 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap" 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.492 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.493 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.494 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.495 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2023-11-17T00:19:06.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-17T00:19:06.497 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.498 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.499 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2023-11-17T00:19:06.500 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.501 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.502 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.503 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.504 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.505 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_password": { 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_password", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web password", 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_user": { 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_user", 2023-11-17T00:19:06.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web user", 2023-11-17T00:19:06.508 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.509 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.510 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:06.511 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2023-11-17T00:19:06.512 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.513 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2023-11-17T00:19:06.514 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-17T00:19:06.515 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.516 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-17T00:19:06.520 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.521 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2023-11-17T00:19:06.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2023-11-17T00:19:06.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.522 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-17T00:19:06.527 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.527 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.527 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.527 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2023-11-17T00:19:06.527 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.529 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-17T00:19:06.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.531 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.532 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2023-11-17T00:19:06.533 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2023-11-17T00:19:06.534 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.535 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.536 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.537 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2023-11-17T00:19:06.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2023-11-17T00:19:06.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2023-11-17T00:19:06.541 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.542 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.543 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-17T00:19:06.544 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2023-11-17T00:19:06.545 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.546 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2023-11-17T00:19:06.547 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.548 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2023-11-17T00:19:06.549 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.550 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2023-11-17T00:19:06.551 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.552 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-17T00:19:06.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.558 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.559 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.560 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.561 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2023-11-17T00:19:06.562 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.563 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.564 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.565 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.568 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.569 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.570 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.571 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.572 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.573 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.574 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-17T00:19:06.576 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.580 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.581 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.582 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.583 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2023-11-17T00:19:06.584 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.585 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-17T00:19:06.586 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.587 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_password": { 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_password", 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.588 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web password", 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_user": { 2023-11-17T00:19:06.589 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_user", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web user", 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2023-11-17T00:19:06.591 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-17T00:19:06.592 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.593 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-17T00:19:06.594 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.595 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2023-11-17T00:19:06.600 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.601 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.602 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-17T00:19:06.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.604 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.605 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.606 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.607 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.608 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2023-11-17T00:19:06.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.611 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.612 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.613 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.614 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.615 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.617 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.619 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.620 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.621 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2023-11-17T00:19:06.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.625 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.626 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.627 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-17T00:19:06.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-17T00:19:06.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.628 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.628 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.628 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.629 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.630 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-17T00:19:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.632 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.633 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.634 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.635 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.636 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-17T00:19:06.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.639 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.640 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.641 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.642 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.643 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.647 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.648 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.649 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.650 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2023-11-17T00:19:06.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-17T00:19:06.654 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.655 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.656 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.661 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-17T00:19:06.663 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.664 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.665 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.666 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.668 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.669 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.670 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.671 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-17T00:19:06.672 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.679 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-17T00:19:06.681 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.682 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.683 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.684 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-17T00:19:06.685 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.686 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.687 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-17T00:19:06.688 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.689 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.692 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-17T00:19:06.693 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.694 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.695 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-17T00:19:06.698 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.699 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.700 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.701 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:06.702 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.703 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.709 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:06.710 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.711 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.713 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.716 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.717 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.718 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.719 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.720 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.721 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.722 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:06.724 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.725 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:06.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.730 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.730 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.730 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2023-11-17T00:19:06.730 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:06.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.733 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.734 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2023-11-17T00:19:06.735 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.738 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.739 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.740 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.741 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.742 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.743 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.744 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.746 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.747 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.748 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.749 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.750 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.751 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2023-11-17T00:19:06.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.758 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.760 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2023-11-17T00:19:06.761 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.764 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.765 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.766 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.768 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.769 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.772 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2023-11-17T00:19:06.774 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.775 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.776 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.777 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.778 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2023-11-17T00:19:06.780 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-17T00:19:06.781 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.782 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.785 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.786 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.787 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.788 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.789 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.790 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.791 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:06.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:06.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2023-11-17T00:19:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2023-11-17T00:19:06.804 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.806 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.807 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.808 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.809 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.812 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.815 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.816 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.817 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.818 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.820 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.821 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.822 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2023-11-17T00:19:06.824 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.825 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-17T00:19:06.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.829 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.831 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.832 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.833 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.834 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.835 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.836 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.837 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2023-11-17T00:19:06.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.840 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.844 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.845 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2023-11-17T00:19:06.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.849 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2023-11-17T00:19:06.850 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2023-11-17T00:19:06.851 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.852 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.853 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.854 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.855 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.857 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.858 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.859 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.860 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.861 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.863 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.864 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.866 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.869 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.870 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.872 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.873 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.874 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.876 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.878 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.879 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.882 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.883 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.885 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.886 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.887 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.888 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.889 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.890 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.891 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.892 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.893 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.894 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.895 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.896 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.897 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.898 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2023-11-17T00:19:06.899 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.900 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.901 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.902 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.903 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.904 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.907 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.908 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.909 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.910 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.911 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.912 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.913 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.915 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.916 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.917 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.918 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.919 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.921 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.922 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.923 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.924 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.925 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.926 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.927 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.928 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "noautoscale": { 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "noautoscale", 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.929 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "global autoscale flag", 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.930 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.933 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.934 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.937 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.937 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2023-11-17T00:19:06.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2023-11-17T00:19:06.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.939 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.940 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.941 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.942 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.943 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.944 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.945 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.946 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.947 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.948 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2023-11-17T00:19:06.950 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.951 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.952 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.953 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.954 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-17T00:19:06.957 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.958 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.959 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.960 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.961 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.962 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.963 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.964 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.965 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.966 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.967 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.968 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.969 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.970 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.971 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.972 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:06.973 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-17T00:19:06.974 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.975 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.976 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.977 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.978 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:06.979 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.981 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.982 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.983 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.984 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.985 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.986 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:06.987 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.988 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.989 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.990 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.991 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.992 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2023-11-17T00:19:06.993 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.994 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2023-11-17T00:19:06.995 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:06.996 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.997 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:06.998 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:06.999 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.000 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.001 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.002 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.003 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.004 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.005 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.007 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.008 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.009 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.010 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.011 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.012 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.013 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.014 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.015 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.016 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.017 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.018 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.019 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.020 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.021 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2023-11-17T00:19:07.022 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.023 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.024 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.025 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.026 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.027 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.028 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.029 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.030 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.031 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.032 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.033 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.034 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.035 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.036 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.037 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.038 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.039 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.040 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.041 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.042 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.043 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.044 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.045 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2023-11-17T00:19:07.046 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.047 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.048 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.049 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.050 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2023-11-17T00:19:07.051 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.054 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.055 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2023-11-17T00:19:07.056 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.057 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.058 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.059 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.060 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.061 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.062 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.063 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2023-11-17T00:19:07.064 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2023-11-17T00:19:07.065 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.066 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.067 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.068 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.069 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.070 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.071 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.072 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.073 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.074 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.075 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.076 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.077 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.078 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.079 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.080 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.081 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.082 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.083 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.084 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.085 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.086 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.087 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.088 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.089 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.090 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.091 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.092 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.093 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.094 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.095 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.096 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.098 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.099 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.100 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.101 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.102 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.103 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.104 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.105 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.106 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.107 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.108 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2023-11-17T00:19:07.109 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.110 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.112 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2023-11-17T00:19:07.113 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.114 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2023-11-17T00:19:07.115 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-17T00:19:07.116 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.117 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.118 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.119 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.121 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.122 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.123 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:07.124 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.125 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.126 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.127 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.128 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.130 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.131 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.132 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.133 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.134 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.135 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.136 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.137 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.138 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.139 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.140 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.141 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.142 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.143 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.144 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.145 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.146 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.147 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.148 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.149 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.150 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.151 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.152 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.153 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.154 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.155 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.156 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2023-11-17T00:19:07.157 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.158 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.159 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.160 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.161 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.162 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.163 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.164 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.165 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.166 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.168 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.169 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.170 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2023-11-17T00:19:07.171 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.172 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.173 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.174 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.175 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.176 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.177 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.178 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.179 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.180 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.181 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.182 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.183 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.184 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2023-11-17T00:19:07.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2023-11-17T00:19:07.187 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.188 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.189 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.190 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: "modules": [ 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2023-11-17T00:19:07.191 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard", 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "iostat", 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "nfs", 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "restful", 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "snap_schedule" 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "available_modules": [ 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.192 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alerts", 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.193 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How frequently to reexamine health status", 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.194 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.195 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.196 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.197 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.198 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.199 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.200 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.201 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.203 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_destination": { 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_destination", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.204 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email address to send alerts to", 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_from_name": { 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_from_name", 2023-11-17T00:19:07.205 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "Ceph", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Email From: name", 2023-11-17T00:19:07.206 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_host": { 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_host", 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.207 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP server", 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.208 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_password": { 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_password", 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.209 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Password to authenticate with", 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_port": { 2023-11-17T00:19:07.210 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_port", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "465", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.211 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP port", 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_sender": { 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_sender", 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.212 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SMTP envelope sender", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.213 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_ssl": { 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_ssl", 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.214 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.215 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "smtp_user": { 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "smtp_user", 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.216 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "User to authenticate as", 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "balancer", 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "active": { 2023-11-17T00:19:07.218 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "active", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically balance PGs across cluster", 2023-11-17T00:19:07.219 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_time": { 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_time", 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.220 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0000", 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "beginning time of day to automatically balance", 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.221 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "begin_weekday": { 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "begin_weekday", 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.222 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.223 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_max_iterations": { 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_max_iterations", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "25", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "250", 2023-11-17T00:19:07.224 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_metrics": { 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_metrics", 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.225 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "pgs,objects,bytes", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-17T00:19:07.226 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_compat_step": { 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crush_compat_step", 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.227 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.5", 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0.001", 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "0.999", 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "aggressiveness of optimization", 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "end_time": { 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_time", 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2359", 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.229 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ending time of day to automatically balance", 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "end_weekday": { 2023-11-17T00:19:07.230 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "end_weekday", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "0", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "6", 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.231 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.232 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.233 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.234 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.235 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.237 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.238 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.240 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "min_score": { 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_score", 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.241 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.242 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "upmap", 2023-11-17T00:19:07.243 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "crush-compat", 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "none", 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap" 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Balancer mode", 2023-11-17T00:19:07.244 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_ids": { 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_ids", 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.245 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.246 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.247 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_deviation": { 2023-11-17T00:19:07.248 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_deviation", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.249 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "upmap_max_optimizations": { 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "upmap_max_optimizations", 2023-11-17T00:19:07.250 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-17T00:19:07.251 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cephadm", 2023-11-17T00:19:07.252 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_down_multiplier": { 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_down_multiplier", 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:07.253 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.254 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_refresh_rate": { 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_refresh_rate", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "20", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.255 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "agent_starting_port": { 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "agent_starting_port", 2023-11-17T00:19:07.256 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4721", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-17T00:19:07.257 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_password": { 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_password", 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.258 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web password", 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.259 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "alertmanager_web_user": { 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "alertmanager_web_user", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.260 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Alertmanager web user", 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_ptrace": { 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_ptrace", 2023-11-17T00:19:07.261 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-17T00:19:07.262 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_interval": { 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_interval", 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.263 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.264 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "autotune_memory_target_ratio": { 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "autotune_memory_target_ratio", 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0.7", 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.265 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.266 INFO:teuthology.orchestra.run.smithi006.stdout: "cgroups_split": { 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cgroups_split", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.267 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "config_checks_enabled": { 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_checks_enabled", 2023-11-17T00:19:07.268 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-17T00:19:07.269 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "config_dashboard": { 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_dashboard", 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.270 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.271 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_alertmanager": { 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_alertmanager", 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.272 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_base": { 2023-11-17T00:19:07.273 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_base", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.274 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Container image name, without the tag", 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_elasticsearch": { 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_elasticsearch", 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.275 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "elasticsearch container image", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.276 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_grafana": { 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_grafana", 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-17T00:19:07.277 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.278 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_haproxy": { 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_haproxy", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.279 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "HAproxy container image", 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_agent": { 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_agent", 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.280 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger agent container image", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.281 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_collector": { 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_collector", 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-17T00:19:07.282 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger collector container image", 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.283 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_jaeger_query": { 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_jaeger_query", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.284 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Jaeger query container image", 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_keepalived": { 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_keepalived", 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.285 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Keepalived container image", 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.286 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_loki": { 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_loki", 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-17T00:19:07.287 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Loki container image", 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.288 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_node_exporter": { 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_node_exporter", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.289 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_prometheus": { 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_prometheus", 2023-11-17T00:19:07.290 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus container image", 2023-11-17T00:19:07.291 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_promtail": { 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_promtail", 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.292 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Promtail container image", 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.293 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image_snmp_gateway": { 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_image_snmp_gateway", 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.294 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "SNMP Gateway container image", 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.295 INFO:teuthology.orchestra.run.smithi006.stdout: "container_init": { 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "container_init", 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.297 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Run podman/docker with `--init`", 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.298 INFO:teuthology.orchestra.run.smithi006.stdout: "daemon_cache_timeout": { 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "daemon_cache_timeout", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.299 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "default_cephadm_command_timeout": { 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_cephadm_command_timeout", 2023-11-17T00:19:07.300 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "900", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-17T00:19:07.301 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "default_registry": { 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "default_registry", 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.302 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "docker.io", 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "device_cache_timeout": { 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_cache_timeout", 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1800", 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.304 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache device inventory", 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "device_enhanced_scan": { 2023-11-17T00:19:07.305 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_enhanced_scan", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "facts_cache_timeout": { 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "facts_cache_timeout", 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.307 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "seconds to cache host facts data", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.308 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "host_check_interval": { 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "host_check_interval", 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:07.309 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to perform a host check", 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.311 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.312 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "log_refresh_metadata": { 2023-11-17T00:19:07.313 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_refresh_metadata", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.314 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.315 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-17T00:19:07.316 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.317 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.318 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.319 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.320 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-17T00:19:07.321 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.322 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.323 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "*", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.324 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "max_count_per_host": { 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_count_per_host", 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.325 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of daemons per service per host", 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.326 INFO:teuthology.orchestra.run.smithi006.stdout: "max_osd_draining_count": { 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_osd_draining_count", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.327 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "migration_current": { 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "migration_current", 2023-11-17T00:19:07.328 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "internal - do not modify", 2023-11-17T00:19:07.329 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": { 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mode", 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.330 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "root", 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm-package", 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "root" 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.331 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "mode for remote execution of cephadm", 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_alerts_path": { 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_alerts_path", 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.333 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_password": { 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_password", 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.334 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web password", 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.335 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "prometheus_web_user": { 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus_web_user", 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.336 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Prometheus web user", 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_insecure": { 2023-11-17T00:19:07.337 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_insecure", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.338 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_password": { 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_password", 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.339 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.340 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_url": { 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_url", 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.341 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.342 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "registry_username": { 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "registry_username", 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.343 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "secure_monitoring_stack": { 2023-11-17T00:19:07.344 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "secure_monitoring_stack", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.345 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "service_discovery_port": { 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "service_discovery_port", 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.346 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8765", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "cephadm service discovery port", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.347 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "ssh_config_file": { 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssh_config_file", 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.348 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.349 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "use_agent": { 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_agent", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.350 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "use_repo_digest": { 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "use_repo_digest", 2023-11-17T00:19:07.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-17T00:19:07.352 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_failed_host_check": { 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_failed_host_check", 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.353 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if the host check fails", 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.354 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_daemons": { 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_daemons", 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.355 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_on_stray_hosts": { 2023-11-17T00:19:07.356 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_on_stray_hosts", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.357 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.358 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crash", 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.359 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.360 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.361 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.362 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.363 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.364 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.365 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.366 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.367 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "retain_interval": { 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retain_interval", 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.368 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "31536000", 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.369 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_recent_interval": { 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_recent_interval", 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1209600", 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.370 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.371 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dashboard", 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-17T00:19:07.372 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.373 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_HOST": { 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.374 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.376 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_ENABLED": { 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_ENABLED", 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.378 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.379 INFO:teuthology.orchestra.run.smithi006.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "ENABLE_BROWSABLE_API": { 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-17T00:19:07.381 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.383 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.387 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-17T00:19:07.388 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.389 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.390 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.391 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.392 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.393 INFO:teuthology.orchestra.run.smithi006.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.394 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-17T00:19:07.395 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.396 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_PASSWORD": { 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.397 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.398 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.399 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.400 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_URL": { 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_URL", 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.401 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.402 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_API_USERNAME": { 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.403 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-17T00:19:07.404 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.405 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.406 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.407 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.408 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.409 INFO:teuthology.orchestra.run.smithi006.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_HOST": { 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-17T00:19:07.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.412 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.413 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.414 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.415 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.416 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.417 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-17T00:19:07.418 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.419 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.420 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.421 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.422 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-17T00:19:07.423 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.424 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-17T00:19:07.425 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.426 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_ENABLED": { 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.427 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.429 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.430 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.431 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-17T00:19:07.432 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.433 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.434 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "45", 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.435 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ACCESS_KEY": { 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.436 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-17T00:19:07.437 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "admin", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.438 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SECRET_KEY": { 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.439 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.440 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "RGW_API_SSL_VERIFY": { 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.441 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.442 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.443 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-17T00:19:07.444 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.445 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-17T00:19:07.446 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.447 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "cross_origin_url": { 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cross_origin_url", 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.448 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.449 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "crt_file": { 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "crt_file", 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.450 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.451 INFO:teuthology.orchestra.run.smithi006.stdout: "debug": { 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "debug", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.452 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Enable/disable debug options", 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "jwt_token_ttl": { 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "jwt_token_ttl", 2023-11-17T00:19:07.453 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "28800", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.454 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.455 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.457 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.458 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.459 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.460 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.461 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.462 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.464 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "motd": { 2023-11-17T00:19:07.465 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "motd", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "The message of the day", 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect_resolve_ip_addr": { 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "redirect_resolve_ip_addr", 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.469 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.470 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8080", 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.471 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:07.472 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.473 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl_server_port": { 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl_server_port", 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.474 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8443", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.475 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.476 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "redirect", 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "redirect" 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.477 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.478 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.479 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "url_prefix": { 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url_prefix", 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.480 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.481 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.482 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_monitoring": { 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_monitoring", 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.483 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "monitor device health metrics", 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.484 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.485 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.486 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.487 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.488 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.489 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.490 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.492 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "mark_out_threshold": { 2023-11-17T00:19:07.493 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mark_out_threshold", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "2419200", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.494 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "pool_name": { 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pool_name", 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.495 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "device_health_metrics", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.496 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "retention_period": { 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "retention_period", 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.497 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15552000", 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long to retain device health metrics", 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.498 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_frequency": { 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_frequency", 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:07.499 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.500 INFO:teuthology.orchestra.run.smithi006.stdout: "self_heal": { 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "self_heal", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.501 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:07.502 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how frequently to wake up and check device health", 2023-11-17T00:19:07.503 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "warn_threshold": { 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "warn_threshold", 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.504 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7257600", 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.505 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "diskprediction_local", 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.506 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.507 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.508 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.509 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.510 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.511 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.512 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.513 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.514 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.515 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "predict_interval": { 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predict_interval", 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.516 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "86400", 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.517 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "predictor_model": { 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "predictor_model", 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "prophetstor", 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.518 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:07.519 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "600", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.520 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.521 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "influx", 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": false, 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "influxdb python module not found", 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "batch_size": { 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "batch_size", 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.522 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5000", 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.523 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "database": { 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "database", 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.524 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "ceph", 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.525 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": { 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "hostname", 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.526 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server hostname", 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.527 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "30", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "5", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.528 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.529 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.531 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.533 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.534 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.535 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.536 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.537 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "password": { 2023-11-17T00:19:07.538 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "password", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.539 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "password of InfluxDB server user", 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "port": { 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "port", 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.540 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "8086", 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.541 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "InfluxDB server port", 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "ssl": { 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ssl", 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.542 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "false", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.543 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "threads": { 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threads", 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.544 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "32", 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.545 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "username": { 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "username", 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.546 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.547 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.547 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "username of InfluxDB server user", 2023-11-17T00:19:07.549 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.549 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.549 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "verify_ssl": { 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "verify_ssl", 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "true", 2023-11-17T00:19:07.550 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.551 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "insights", 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.552 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.553 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.554 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.555 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.556 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.557 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.558 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.559 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.560 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "iostat", 2023-11-17T00:19:07.561 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.562 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.563 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.564 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.565 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.566 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.567 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.568 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.570 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "k8sevents", 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.571 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_event_retention_days": { 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "ceph_event_retention_days", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "7", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.572 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "config_check_secs": { 2023-11-17T00:19:07.573 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "config_check_secs", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "10", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-17T00:19:07.574 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.575 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.576 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.579 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.580 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.581 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.582 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.583 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "localpool", 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.584 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "failure_domain": { 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "failure_domain", 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "host", 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.585 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "failure domain for any created local pool", 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.586 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.587 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.589 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.590 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.591 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.592 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.593 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.594 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "min_size": { 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "min_size", 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.595 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default min_size for any created local pool", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-17T00:19:07.596 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "num_rep": { 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "num_rep", 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.597 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3", 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default replica count for any created local pool", 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_num": { 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_num", 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "128", 2023-11-17T00:19:07.599 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "default pg_num for any created local pool", 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.600 INFO:teuthology.orchestra.run.smithi006.stdout: "prefix": { 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prefix", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "name prefix for any created local pool", 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "subtree": { 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "subtree", 2023-11-17T00:19:07.602 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "rack", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-17T00:19:07.603 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mds_autoscaler", 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.604 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.605 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.606 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.607 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.608 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.609 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.610 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.611 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.612 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirroring", 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.614 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.615 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.616 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.617 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.618 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.619 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.620 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.622 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "nfs", 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.624 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.625 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.626 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.627 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.628 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.629 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.630 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.631 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.632 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:07.633 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "fail_fs": { 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "fail_fs", 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.634 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.636 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.637 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.638 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.639 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.640 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.641 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.642 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.643 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.644 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator": { 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "orchestrator", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.645 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm", 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "rook", 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "test_orchestrator" 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Orchestrator backend", 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.646 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_perf_query", 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.647 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.648 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.649 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.650 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.651 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.652 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.653 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.654 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.655 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.656 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "osd_support", 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.657 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.658 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.659 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.660 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.661 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.664 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.665 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "pg_autoscaler", 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.666 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.667 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.668 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.669 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.670 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.671 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.672 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.673 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.675 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "noautoscale": { 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "noautoscale", 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.676 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "global autoscale flag", 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:07.677 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.678 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "threshold": { 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "threshold", 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.679 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "3.0", 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1.0", 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "scaling threshold", 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.680 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "progress", 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.681 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_pg_recovery_event": { 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_pg_recovery_event", 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.682 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:07.683 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.684 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.685 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.686 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.687 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.688 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.689 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.690 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.691 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.693 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "max_completed_events": { 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_completed_events", 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.694 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "50", 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "number of past completed events to remember", 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.695 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "sleep_interval": { 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "sleep_interval", 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "5", 2023-11-17T00:19:07.696 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "how long the module is going to sleep", 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "prometheus", 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "cache": { 2023-11-17T00:19:07.698 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "cache", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.699 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "exclude_perf_counters": { 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "exclude_perf_counters", 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.700 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-17T00:19:07.701 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.702 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.703 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.704 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.705 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.706 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.707 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.708 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.709 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.710 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools": { 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-17T00:19:07.712 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.713 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "scrape_interval": { 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "scrape_interval", 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.714 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15.0", 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.715 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "::", 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.716 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.717 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "9283", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.718 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "stale_cache_strategy": { 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stale_cache_strategy", 2023-11-17T00:19:07.719 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "log", 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.720 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_behaviour": { 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_behaviour", 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.721 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "default", 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "default", 2023-11-17T00:19:07.722 INFO:teuthology.orchestra.run.smithi006.stdout: "error" 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "standby_error_status_code": { 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "standby_error_status_code", 2023-11-17T00:19:07.723 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "500", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "400", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "599", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.724 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.725 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.726 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.727 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.729 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.730 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.731 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.733 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.734 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_snap_create": { 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_snap_create", 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.735 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10", 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.736 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "mirror_snapshot_schedule": { 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mirror_snapshot_schedule", 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.737 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.738 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "trash_purge_schedule": { 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "trash_purge_schedule", 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.739 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "restful", 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_auth": { 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enable_auth", 2023-11-17T00:19:07.741 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.742 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "key_file": { 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "key_file", 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.743 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.744 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.745 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.746 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.747 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.748 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.749 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.750 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.751 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.751 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.751 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.751 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.752 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.753 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "server_addr": { 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_addr", 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.754 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.755 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "server_port": { 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "server_port", 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.757 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rgw", 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.758 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.759 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.760 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.761 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.762 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.763 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.764 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.765 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.766 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.767 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rook", 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "drive_group_interval": { 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "drive_group_interval", 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.768 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "300.0", 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-17T00:19:07.769 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.770 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.772 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.773 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.774 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.775 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.776 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.777 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.778 INFO:teuthology.orchestra.run.smithi006.stdout: "storage_class": { 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "storage_class", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "local", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.779 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "selftest", 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.781 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.782 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.783 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.784 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.785 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.786 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.787 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.789 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "roption1": { 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption1", 2023-11-17T00:19:07.790 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.791 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "roption2": { 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "roption2", 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.792 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "xyz", 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.793 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption1": { 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption1", 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.794 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.795 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption2": { 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption2", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.796 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption3": { 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption3", 2023-11-17T00:19:07.797 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "float", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.798 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption4": { 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption4", 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.799 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption5": { 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption5", 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.801 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption6": { 2023-11-17T00:19:07.802 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption6", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.803 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "rwoption7": { 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rwoption7", 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.804 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "1", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "42", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.805 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "testkey": { 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testkey", 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.806 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "testlkey": { 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testlkey", 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.808 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "testnewline": { 2023-11-17T00:19:07.809 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "testnewline", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.810 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.811 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "allow_m_granularity": { 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "allow_m_granularity", 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.812 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "allow minute scheduled snapshots", 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.813 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "dump_on_update": { 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "dump_on_update", 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.814 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "dump database to debug log on update", 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.815 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.816 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.817 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.818 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.819 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.820 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.821 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.822 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.823 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.824 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "stats", 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.825 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.826 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.827 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.828 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.830 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.831 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.833 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.834 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "status", 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.835 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.836 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.837 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.838 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.839 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.840 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.841 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.842 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.843 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telegraf", 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.844 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "address": { 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "address", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.845 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.846 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "15", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.847 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.848 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.849 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.850 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.851 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.852 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.853 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.854 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.856 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "telemetry", 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_basic": { 2023-11-17T00:19:07.857 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_basic", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.858 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_crash": { 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_crash", 2023-11-17T00:19:07.859 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-17T00:19:07.860 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_device": { 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_device", 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.861 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "True", 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.862 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_ident": { 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_ident", 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.863 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.864 INFO:teuthology.orchestra.run.smithi006.stdout: "channel_perf": { 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "channel_perf", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.865 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "contact": { 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "contact", 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.866 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.867 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "description": { 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "description", 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.868 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "device_url": { 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "device_url", 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.870 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": { 2023-11-17T00:19:07.871 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "enabled", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.872 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.873 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "24", 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "8", 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.874 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "last_opt_revision": { 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "last_opt_revision", 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.875 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "1", 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.876 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard": { 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard", 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.877 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.878 INFO:teuthology.orchestra.run.smithi006.stdout: "leaderboard_description": { 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "leaderboard_description", 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.879 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.880 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.881 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.882 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.883 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.884 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.885 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.886 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.887 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.888 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "organization": { 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "organization", 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.889 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.890 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "proxy": { 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "proxy", 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.892 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "url": { 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "url", 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-17T00:19:07.893 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.894 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "test_orchestrator", 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.895 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.896 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.897 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.898 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.899 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.900 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.901 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.903 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.905 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.906 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.907 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.908 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.909 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.910 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.911 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.912 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.913 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.914 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "max_concurrent_clones": { 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "max_concurrent_clones", 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "4", 2023-11-17T00:19:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.916 INFO:teuthology.orchestra.run.smithi006.stdout: "snapshot_clone_delay": { 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snapshot_clone_delay", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "0", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.917 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.918 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix", 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "can_run": true, 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "error_string": "", 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "module_options": { 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "discovery_interval": { 2023-11-17T00:19:07.919 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "discovery_interval", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "uint", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "100", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.920 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "identifier": { 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "identifier", 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.921 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.922 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "interval": { 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "interval", 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "secs", 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.923 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "60", 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.924 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "log_level": { 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_level", 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.925 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.926 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster": { 2023-11-17T00:19:07.927 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.928 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_cluster_level": { 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_cluster_level", 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.929 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "info", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [ 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "critical", 2023-11-17T00:19:07.930 INFO:teuthology.orchestra.run.smithi006.stdout: "debug", 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "error", 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "info", 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "warning" 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.931 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": { 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "log_to_file", 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "bool", 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 1, 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "False", 2023-11-17T00:19:07.932 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.933 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_host": { 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_host", 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "", 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.934 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_port": { 2023-11-17T00:19:07.935 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_port", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "int", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "10051", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.936 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "zabbix_sender": { 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "zabbix_sender", 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "str", 2023-11-17T00:19:07.937 INFO:teuthology.orchestra.run.smithi006.stdout: "level": "advanced", 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "flags": 0, 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "min": "", 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "max": "", 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "enum_allowed": [], 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "desc": "", 2023-11-17T00:19:07.938 INFO:teuthology.orchestra.run.smithi006.stdout: "long_desc": "", 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: "see_also": [] 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: "services": { 2023-11-17T00:19:07.939 INFO:teuthology.orchestra.run.smithi006.stdout: "dashboard": "https://172.21.15.6:8443/" 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "always_on_modules": { 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "octopus": [ 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2023-11-17T00:19:07.940 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.941 INFO:teuthology.orchestra.run.smithi006.stdout: "pacific": [ 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2023-11-17T00:19:07.942 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "quincy": [ 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2023-11-17T00:19:07.943 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-17T00:19:07.944 INFO:teuthology.orchestra.run.smithi006.stdout: "reef": [ 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "balancer", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "crash", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "devicehealth", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "orchestrator", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "pg_autoscaler", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "progress", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "rbd_support", 2023-11-17T00:19:07.945 INFO:teuthology.orchestra.run.smithi006.stdout: "status", 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: "telemetry", 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: "volumes" 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: "last_failure_osd_epoch": 93, 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: "active_clients": [ 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.946 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "devicehealth", 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1732681901 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.947 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "libcephsqlite", 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 176394000 2023-11-17T00:19:07.948 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "rbd_support", 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2738554464 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.950 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3212038614 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.951 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "snap_schedule", 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 4247753561 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.952 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "volumes", 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: "addrvec": [ 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v2", 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "172.21.15.6:0", 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3245201296 2023-11-17T00:19:07.953 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.954 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.954 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:07.954 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-17T00:19:07.954 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:19:07.956 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:04 smithi186 ceph-mon[131487]: pgmap v315: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:07.956 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:04 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3892768086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:19:07.956 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:04 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/3663365122' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-17T00:19:07.957 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:06 smithi186 ceph-mon[131487]: pgmap v316: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-17T00:19:07.957 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:04 smithi087 ceph-mon[131182]: pgmap v315: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:07.957 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:04 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3892768086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:19:07.957 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:04 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/3663365122' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-17T00:19:07.957 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:06 smithi087 ceph-mon[131182]: pgmap v316: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 msgr2=0x7f30f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 0x7f30f40a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005950 tx=0x7f30fc00b500 comp rx=0 tx=0).stop 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 msgr2=0x7f310c19bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 0x7f310c19bfe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3104009fd0 tx=0x7f310400ce50 comp rx=0 tx=0).stop 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 shutdown_connections 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f30f40a12c0 0x7f30f40a3770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:07.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f310c1084e0 0x7f310c19bfe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:07.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1024d0 0x7f310c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:07.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 --2- 172.21.15.6:0/3663365122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f310c102e80 0x7f310c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:19:07.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 >> 172.21.15.6:0/3663365122 conn(0x7f310c0fdf90 msgr2=0x7f310c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:19:07.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 shutdown_connections 2023-11-17T00:19:07.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:19:04.526+0000 7f3110fff700 1 -- 172.21.15.6:0/3663365122 wait complete. 2023-11-17T00:19:07.966 DEBUG:tasks.check_counter:Getting stats from a 2023-11-17T00:19:07.966 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-11-17T00:19:07.967 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:04 smithi006 ceph-mon[97840]: pgmap v315: 129 pgs: 7 active+clean+snaptrim_wait, 120 active+clean, 2 active+clean+snaptrim; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-17T00:19:07.967 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:04 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3892768086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:19:07.967 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:04 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/3663365122' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-17T00:19:07.967 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:06 smithi006 ceph-mon[97840]: pgmap v316: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-17T00:19:08.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:07 smithi006 ceph-mon[97840]: osdmap e101: 12 total, 12 up, 12 in 2023-11-17T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:07 smithi087 ceph-mon[131182]: osdmap e101: 12 total, 12 up, 12 in 2023-11-17T00:19:08.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:07 smithi186 ceph-mon[131487]: osdmap e101: 12 total, 12 up, 12 in 2023-11-17T00:19:09.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:08 smithi006 ceph-mon[97840]: pgmap v318: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2023-11-17T00:19:09.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:08 smithi006 ceph-mon[97840]: osdmap e102: 12 total, 12 up, 12 in 2023-11-17T00:19:09.359 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:08 smithi087 ceph-mon[131182]: pgmap v318: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2023-11-17T00:19:09.365 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:08 smithi087 ceph-mon[131182]: osdmap e102: 12 total, 12 up, 12 in 2023-11-17T00:19:09.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:08 smithi186 ceph-mon[131487]: pgmap v318: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2023-11-17T00:19:09.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:08 smithi186 ceph-mon[131487]: osdmap e102: 12 total, 12 up, 12 in 2023-11-17T00:19:09.408 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:19:11.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:10 smithi006 ceph-mon[97840]: pgmap v320: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 35 KiB/s wr, 5 op/s 2023-11-17T00:19:11.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:10 smithi087 ceph-mon[131182]: pgmap v320: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 35 KiB/s wr, 5 op/s 2023-11-17T00:19:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:10 smithi186 ceph-mon[131487]: pgmap v320: 129 pgs: 129 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 35 KiB/s wr, 5 op/s 2023-11-17T00:19:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:12 smithi006 ceph-mon[97840]: pgmap v321: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 4.9 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 35 KiB/s wr, 6 op/s 2023-11-17T00:19:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:12 smithi087 ceph-mon[131182]: pgmap v321: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 4.9 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 35 KiB/s wr, 6 op/s 2023-11-17T00:19:13.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:12 smithi186 ceph-mon[131487]: pgmap v321: 129 pgs: 22 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 40 active+clean; 4.9 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 35 KiB/s wr, 6 op/s 2023-11-17T00:19:14.082 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:19:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:14.082 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.004059715, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000891020, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:14.083 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:14.084 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 229, 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 216, 2023-11-17T00:19:14.085 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1561088, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 122144, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.057147485, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.021304814, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.040635827, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.005760084, 2023-11-17T00:19:14.086 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 216, 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.028125463, 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000130210 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:14.087 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1561088, 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 122144 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 163, 2023-11-17T00:19:14.088 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 180288, 2023-11-17T00:19:14.089 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.021377128, 2023-11-17T00:19:14.090 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.014630313, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003141577, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 160, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.025639896, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000160249 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:14.091 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 180288 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2023-11-17T00:19:14.092 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:14.093 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:14.094 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:14.095 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:14.096 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:14.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148977, 2023-11-17T00:19:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2023-11-17T00:19:14.098 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2023-11-17T00:19:14.109 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:14.110 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2023-11-17T00:19:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2023-11-17T00:19:14.114 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2023-11-17T00:19:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 227, 2023-11-17T00:19:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1526381, 2023-11-17T00:19:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 227, 2023-11-17T00:19:14.115 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 227, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1526381, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:14.116 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:14.152 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:14.152 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:19:14.865 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-11-17T00:19:14.866 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-11-17T00:19:14.866 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-11-17T00:19:14.866 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-11-17T00:19:14.866 DEBUG:tasks.check_counter:Getting stats from d 2023-11-17T00:19:14.866 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-11-17T00:19:15.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:14 smithi006 ceph-mon[97840]: pgmap v322: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-11-17T00:19:15.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:14 smithi087 ceph-mon[131182]: pgmap v322: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-11-17T00:19:15.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:14 smithi186 ceph-mon[131487]: pgmap v322: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-11-17T00:19:16.181 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:19:17.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:16 smithi006 ceph-mon[97840]: pgmap v323: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2023-11-17T00:19:17.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:16 smithi087 ceph-mon[131182]: pgmap v323: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2023-11-17T00:19:17.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:16 smithi186 ceph-mon[131487]: pgmap v323: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 236 B/s wr, 2 op/s 2023-11-17T00:19:18.816 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 235, 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 222, 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1562720, 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 125696, 2023-11-17T00:19:18.817 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.059094620, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.022052552, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.039466970, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.005145962, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 222, 2023-11-17T00:19:18.818 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.019736578, 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000088903 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.819 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1562720, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 125696 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 167, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 180992, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:18.820 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.021732439, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.015238313, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.003210172, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 164, 2023-11-17T00:19:18.821 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.019497381, 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000118886 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:18.822 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 180992 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:18.823 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.000795237, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000678682, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:18.824 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:18.825 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.826 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:18.827 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:18.828 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:18.829 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:18.829 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:18.829 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:18.829 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:18.829 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:18.830 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:18.830 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:18.831 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:18.831 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:18.831 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:18.831 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:18.847 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:18.848 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:18.848 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:18.848 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:18.848 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:18.849 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2023-11-17T00:19:18.849 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:18.849 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:18.850 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:18.850 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:18.850 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148714, 2023-11-17T00:19:18.850 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2023-11-17T00:19:18.851 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:18.851 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:18.851 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2023-11-17T00:19:18.851 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2023-11-17T00:19:18.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:18.879 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:18.880 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2023-11-17T00:19:18.881 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 233, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1527161, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:18.882 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 233, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 233, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1527161, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:18.884 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:18.884 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:18.884 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:19:19.227 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:18 smithi006 ceph-mon[97840]: pgmap v324: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:19:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:18 smithi087 ceph-mon[131182]: pgmap v324: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:19:19.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:18 smithi186 ceph-mon[131487]: pgmap v324: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-11-17T00:19:19.516 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-11-17T00:19:19.516 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-11-17T00:19:19.516 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-11-17T00:19:19.516 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-17T00:19:19.516 DEBUG:tasks.check_counter:Getting stats from g 2023-11-17T00:19:19.517 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-11-17T00:19:20.816 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:19:21.280 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:20 smithi006 ceph-mon[97840]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:19:21.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:20 smithi087 ceph-mon[131182]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:19:21.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:20 smithi186 ceph-mon[131487]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 187 B/s wr, 2 op/s 2023-11-17T00:19:23.218 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:19:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 229, 2023-11-17T00:19:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 218, 2023-11-17T00:19:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1559232, 2023-11-17T00:19:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 124064, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.060227519, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.021519248, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.039951665, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.006656819, 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 217, 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.021870005, 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000100783 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1559232, 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 124064 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 6, 2023-11-17T00:19:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 167, 2023-11-17T00:19:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 3264, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 181088, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.020382355, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.013095815, 2023-11-17T00:19:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.004007011, 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 165, 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.031818310, 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000192838 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 181088 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.000827887, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000694492, 2023-11-17T00:19:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2023-11-17T00:19:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2023-11-17T00:19:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148977, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2023-11-17T00:19:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2023-11-17T00:19:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2023-11-17T00:19:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 231, 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1526901, 2023-11-17T00:19:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 231, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 231, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1526901, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:23.241 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:19:23.257 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:22 smithi006 ceph-mon[97840]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:19:23.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:22 smithi087 ceph-mon[131182]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:19:23.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:22 smithi186 ceph-mon[131487]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2023-11-17T00:19:23.879 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-11-17T00:19:23.880 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-11-17T00:19:23.880 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-11-17T00:19:23.880 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-17T00:19:23.880 DEBUG:tasks.check_counter:Getting stats from j 2023-11-17T00:19:23.881 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-11-17T00:19:25.204 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-17T00:19:25.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:25 smithi087 ceph-mon[131182]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:19:25.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:25 smithi186 ceph-mon[131487]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:19:25.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:25 smithi006 ceph-mon[97840]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-11-17T00:19:27.329 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:27 smithi006 ceph-mon[97840]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:19:37.347 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:27 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:37.395 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.003730924, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.000632361, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:37.396 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:37.419 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 238, 2023-11-17T00:19:37.420 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 225, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 1563536, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 127472, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.056402310, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.020700559, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.041544904, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.005702199, 2023-11-17T00:19:37.421 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 225, 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.016067447, 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000071410 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:37.422 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:37.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.423 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 1563536, 2023-11-17T00:19:37.423 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 127472 2023-11-17T00:19:37.423 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.423 INFO:teuthology.orchestra.run.smithi006.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages": 170, 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_bytes": 181520, 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:37.425 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:37.426 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_total_time": 0.019448345, 2023-11-17T00:19:37.426 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_send_time": 0.013002444, 2023-11-17T00:19:37.426 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_recv_time": 0.002890275, 2023-11-17T00:19:37.426 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:37.426 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:37.488 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 167, 2023-11-17T00:19:37.488 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.028906703, 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000173094 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.489 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: "msgr_send_encrypted_bytes": 181520 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: "cct": { 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: "total_workers": 0, 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: "unhealthy_workers": 0 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.490 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool": { 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:37.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:37.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:37.552 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:37.552 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:37.552 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:37.886 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:37.886 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:37.912 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:37.913 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_items": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_bytes": 148714, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_anon_items": 30, 2023-11-17T00:19:37.914 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:37.915 INFO:teuthology.orchestra.run.smithi006.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:37.915 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_bytes": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_items": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:37.942 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_items": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "pgmap_items": 0, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_co_items": 2, 2023-11-17T00:19:37.943 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_1_items": 0, 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "unittest_2_items": 0 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "val": 0, 2023-11-17T00:19:37.944 INFO:teuthology.orchestra.run.smithi006.stdout: "max": 104857600, 2023-11-17T00:19:37.974 INFO:teuthology.orchestra.run.smithi006.stdout: "get_started": 0, 2023-11-17T00:19:37.985 INFO:teuthology.orchestra.run.smithi006.stdout: "get": 236, 2023-11-17T00:19:37.985 INFO:teuthology.orchestra.run.smithi006.stdout: "get_sum": 1527551, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "get_or_fail_success": 236, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "take": 0, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "take_sum": 0, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "put": 236, 2023-11-17T00:19:37.986 INFO:teuthology.orchestra.run.smithi006.stdout: "put_sum": 1527551, 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: "wait": { 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: "avgcount": 0, 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: "sum": 0.000000000, 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: "avgtime": 0.000000000 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-17T00:19:37.987 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-17T00:19:37.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:27 smithi186 ceph-mon[131487]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:19:37.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:27 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:19:37.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:29 smithi186 ceph-mon[131487]: pgmap v329: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:37.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:29 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:29 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:30 smithi186 ceph-mon[131487]: pgmap v330: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:37.989 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:37.990 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:37.990 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:37.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:37.999 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:31 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.000 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:19:38.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:19:38.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.023 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.024 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.024 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.024 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.024 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:19:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:19:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:32 smithi186 ceph-mon[131487]: pgmap v331: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:34 smithi186 ceph-mon[131487]: pgmap v332: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:34 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:34 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: pgmap v333: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.028 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.028 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.028 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.038 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:19:38.038 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:19:38.038 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:36 smithi186 ceph-mon[131487]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:27 smithi087 ceph-mon[131182]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-11-17T00:19:38.039 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:27 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-17T00:19:38.040 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:29 smithi087 ceph-mon[131182]: pgmap v329: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:38.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:29 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.041 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:29 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:30 smithi087 ceph-mon[131182]: pgmap v330: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.042 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:31 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:19:38.043 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:19:38.044 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:19:38.045 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:32 smithi087 ceph-mon[131182]: pgmap v331: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.045 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:34 smithi087 ceph-mon[131182]: pgmap v332: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:34 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:34 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: pgmap v333: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.046 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:19:38.047 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:36 smithi087 ceph-mon[131182]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.048 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-11-17T00:19:38.048 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-11-17T00:19:38.048 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-11-17T00:19:38.048 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-17T00:19:38.048 DEBUG:tasks.check_counter:Getting stats from b 2023-11-17T00:19:38.049 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-11-17T00:19:38.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:29 smithi006 ceph-mon[97840]: pgmap v329: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:38.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:29 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.050 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:29 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:30 smithi006 ceph-mon[97840]: pgmap v330: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:31 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.052 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.053 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi087 to 196.2M 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi087 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:32 smithi006 ceph-mon[97840]: pgmap v331: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:34 smithi006 ceph-mon[97840]: pgmap v332: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:34 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:34 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: pgmap v333: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:38.054 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-17T00:19:38.055 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:36 smithi006 ceph-mon[97840]: from='mgr.25469 ' entity='mgr.x' 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Service mds.all has no related daemon already placed. 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Service mgr has no related daemon already placed. 2023-11-17T00:19:38.056 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:37 smithi006 ceph-mon[97840]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.110 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:19:38.110 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:19:38.110 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Service mds.all has no related daemon already placed. 2023-11-17T00:19:38.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Service mgr has no related daemon already placed. 2023-11-17T00:19:38.111 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:37 smithi087 ceph-mon[131182]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.124 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Adjusting osd_memory_target on smithi006 to 223.0M 2023-11-17T00:19:38.124 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Unable to set osd_memory_target on smithi006 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-17T00:19:38.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Service mds.all has no related daemon already placed. 2023-11-17T00:19:38.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Service mgr has no related daemon already placed. 2023-11-17T00:19:38.125 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:37 smithi186 ceph-mon[131487]: Service mon has no related daemon already placed. 2023-11-17T00:19:38.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:38 smithi006 ceph-mon[97840]: pgmap v334: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:38.998 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:19:39.106 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:38 smithi087 ceph-mon[131182]: pgmap v334: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:39.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:38 smithi186 ceph-mon[131487]: pgmap v334: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:40.975 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:40 smithi006 ceph-mon[97840]: pgmap v335: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:40.998 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-11-17T00:19:40.998 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 95342, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 94721, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 34520489, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 281728873, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 11, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 5, 2023-11-17T00:19:40.999 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 6.289243863, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 1.701025550, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 1.040134163, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 3.311549410, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 94720, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 148.532833776, 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001568125 2023-11-17T00:19:41.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 921, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000134564, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000146 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1946768, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 251872 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.001 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 90320, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 90186, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 34218206, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 436297030, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 178, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 6, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 6.610506748, 2023-11-17T00:19:41.002 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 2.095425423, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.992353515, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 3.275568053, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 90183, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1120.536877173, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.012425145 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.003 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 377, 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000047016, 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000124 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 86368, 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 144992 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:41.004 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 297678, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 501649, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 108022103, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 455005699, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 14, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 6, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 15.876659123, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 6.286763733, 2023-11-17T00:19:41.005 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 303.827752520, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 3.138574472, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 501648, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1568.782988583, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003127258 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 504285, 2023-11-17T00:19:41.006 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.107100040, 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000212 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 57888, 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 345728 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2023-11-17T00:19:41.007 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 23937, 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 260.639324007, 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.010888554 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 68203, 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 61.831949305, 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000906586 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 105731, 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 105731, 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 105731, 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1080.686823869, 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.010221097 2023-11-17T00:19:41.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 26295, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 368, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 3, 2023-11-17T00:19:41.011 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 88328, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 61810, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 26518, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 27, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 6, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 7, 2023-11-17T00:19:41.012 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 2, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 106304, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 105721, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 0, 2023-11-17T00:19:41.013 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 47300, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2023-11-17T00:19:41.014 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 4, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 246, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 6, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 47300, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 153, 2023-11-17T00:19:41.015 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 14, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 139, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 47147, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 3301, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 105052, 2023-11-17T00:19:41.016 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 1, 2023-11-17T00:19:41.017 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-17T00:19:41.017 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-17T00:19:41.018 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 104037, 2023-11-17T00:19:41.018 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 100541, 2023-11-17T00:19:41.018 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 345, 2023-11-17T00:19:41.018 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 55298 2023-11-17T00:19:41.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 11, 2023-11-17T00:19:41.019 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 50124, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 50113, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2023-11-17T00:19:41.020 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 14, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 499, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2023-11-17T00:19:41.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 244793, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 244753, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 244753, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 40, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 45, 2023-11-17T00:19:41.022 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 352, 2023-11-17T00:19:41.023 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 351, 2023-11-17T00:19:41.023 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 351, 2023-11-17T00:19:41.023 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 2, 2023-11-17T00:19:41.024 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2023-11-17T00:19:41.024 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 1, 2023-11-17T00:19:41.024 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 760544883, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 760649321, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 0, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2424, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1079.217653102, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.445221804 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.025 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 0 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 87382, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 137509, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 50127, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 2587, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 3084, 2023-11-17T00:19:41.026 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 497, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 88328, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 187524, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 99196, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 7, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 50147, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 50140, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 491628, 2023-11-17T00:19:41.027 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 207156 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 109458, 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 105731, 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 36, 2023-11-17T00:19:41.028 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 50012, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 540.944555673, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.010816295 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 147, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.077842579, 2023-11-17T00:19:41.029 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000529541 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2023-11-17T00:19:41.030 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 118, 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.039598084, 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000335576 2023-11-17T00:19:41.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2, 2023-11-17T00:19:41.032 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000266747, 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000133373 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.034 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.035 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2023-11-17T00:19:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 11, 2023-11-17T00:19:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.006852375, 2023-11-17T00:19:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000622943 2023-11-17T00:19:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.036 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 125, 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.124129469, 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000993035 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2023-11-17T00:19:41.037 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 11, 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 13.570589616, 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 1.233689965 2023-11-17T00:19:41.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.039 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2023-11-17T00:19:41.039 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.039 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.039 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.043 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 5129, 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 17.888734237, 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003487762 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2, 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002522031, 2023-11-17T00:19:41.044 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001261015 2023-11-17T00:19:41.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.092 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2023-11-17T00:19:41.092 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.092 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.093 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 116, 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.359047648, 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003095238 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 5, 2023-11-17T00:19:41.094 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.799012686, 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.159802537 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3, 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.002805432, 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000935144 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.095 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4, 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.012384763, 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003096190 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.096 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 22, 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 10.019466814, 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.455430309 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.097 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2023-11-17T00:19:41.098 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.098 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.098 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.106 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2023-11-17T00:19:41.106 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 16, 2023-11-17T00:19:41.106 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.031574742, 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.001973421 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.107 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 50008, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 496.807440973, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.009934559 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.108 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 2, 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 6, 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 4, 2023-11-17T00:19:41.109 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 2, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 4234, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 2117, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 649 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:41.110 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:41.111 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:41.112 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:41.112 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:41.112 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:41.112 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:41.113 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:41.114 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:41.114 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:41.116 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:41.116 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:41.116 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:41.117 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 2083635, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 49836, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 1144, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 13, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2023-11-17T00:19:41.118 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2023-11-17T00:19:41.119 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:41.119 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:41.119 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:41.119 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 9880, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 192, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 256239294, 2023-11-17T00:19:41.120 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 1863075, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2023-11-17T00:19:41.121 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 279877, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 978393171, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 26, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 279877, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 279877, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 168884.647946774, 2023-11-17T00:19:41.122 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.603424532 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 279877, 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 477843 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 279877, 2023-11-17T00:19:41.123 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 47706, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 232171, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 23309, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 101338, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 392, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 2802, 2023-11-17T00:19:41.124 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 202, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 8, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 101379, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2023-11-17T00:19:41.125 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 47314, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 152691, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2023-11-17T00:19:41.126 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 1, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2023-11-17T00:19:41.127 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2023-11-17T00:19:41.128 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 102, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 19, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 12, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 12, 2023-11-17T00:19:41.129 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 0, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 45676, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 0, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 2731 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 1, 2023-11-17T00:19:41.130 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 106258 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2023-11-17T00:19:41.131 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 48, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 24, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 50113, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:41.132 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 483336, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 133191881, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 483336, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:41.133 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 483336, 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 133191881, 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.134 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2023-11-17T00:19:41.135 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 279877, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 985027256, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 279877, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 985027256, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.136 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:41.137 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 279877, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 279877, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 279877, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 279877, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.138 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 50113, 2023-11-17T00:19:41.139 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 6634085, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 50113, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 378, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 6634085, 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:41.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x561b25969cc0": { 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:41.141 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 244793, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 756455017, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 244793, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 2424, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 756455017, 2023-11-17T00:19:41.142 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:41.143 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-11-17T00:19:41.144 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:40 smithi186 ceph-mon[131487]: pgmap v335: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:41.144 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:40 smithi087 ceph-mon[131182]: pgmap v335: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:41.421 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-11-17T00:19:41.421 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-11-17T00:19:41.421 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2023-11-17T00:19:41.421 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=368 2023-11-17T00:19:41.421 DEBUG:tasks.check_counter:Getting stats from e 2023-11-17T00:19:41.422 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-11-17T00:19:42.383 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:19:42.944 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:42 smithi087 ceph-mon[131182]: pgmap v336: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:43.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:42 smithi186 ceph-mon[131487]: pgmap v336: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:43.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:42 smithi006 ceph-mon[97840]: pgmap v336: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:44.070 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-11-17T00:19:44.071 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:44.071 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:44.071 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:44.071 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.000999230, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000096722, 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:44.073 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:44.074 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 241, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 228, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1564352, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 129232, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.043133833, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.012875652, 2023-11-17T00:19:44.075 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.033053967, 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.006025098, 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 228, 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.035198768, 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000154380 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.076 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1564352, 2023-11-17T00:19:44.077 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 129232 2023-11-17T00:19:44.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.078 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:44.078 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:44.078 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 172, 2023-11-17T00:19:44.078 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:44.079 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 181872, 2023-11-17T00:19:44.079 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:44.079 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:44.079 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.010443918, 2023-11-17T00:19:44.079 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.007050337, 2023-11-17T00:19:44.080 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.001281638, 2023-11-17T00:19:44.080 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:44.080 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:44.080 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 169, 2023-11-17T00:19:44.081 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.032541310, 2023-11-17T00:19:44.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000192552 2023-11-17T00:19:44.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.081 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:44.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:44.082 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:44.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:44.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.083 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:44.083 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 181872 2023-11-17T00:19:44.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.083 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2023-11-17T00:19:44.084 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2023-11-17T00:19:44.084 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2023-11-17T00:19:44.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.084 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2023-11-17T00:19:44.085 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:44.085 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:44.085 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:44.085 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:44.086 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:44.086 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:44.086 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:44.087 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:44.088 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:44.089 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:44.090 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2023-11-17T00:19:44.090 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:44.090 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:44.090 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148670, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 30, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2023-11-17T00:19:44.091 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:44.092 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:44.093 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2023-11-17T00:19:44.093 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:44.093 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2023-11-17T00:19:44.093 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:44.093 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2023-11-17T00:19:44.094 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:44.094 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2023-11-17T00:19:44.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:44.094 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:44.094 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 239, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1527941, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 239, 2023-11-17T00:19:44.095 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 239, 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1527941, 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:44.096 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:44.097 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:44.097 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:44.097 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:44.097 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-11-17T00:19:44.483 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-11-17T00:19:44.484 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-11-17T00:19:44.484 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-11-17T00:19:44.484 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2023-11-17T00:19:44.484 DEBUG:tasks.check_counter:Getting stats from h 2023-11-17T00:19:44.485 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-11-17T00:19:44.750 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:44 smithi087 ceph-mon[131182]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:45.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:44 smithi186 ceph-mon[131487]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:45.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:44 smithi006 ceph-mon[97840]: pgmap v337: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:45.455 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:19:47.104 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:46 smithi087 ceph-mon[131182]: pgmap v338: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:47.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:46 smithi186 ceph-mon[131487]: pgmap v338: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:47.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:46 smithi006 ceph-mon[97840]: pgmap v338: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:49.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:48 smithi087 ceph-mon[131182]: pgmap v339: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:49.115 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:48 smithi186 ceph-mon[131487]: pgmap v339: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:49.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:48 smithi006 ceph-mon[97840]: pgmap v339: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:50.432 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-11-17T00:19:50.432 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:50.432 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 4, 2023-11-17T00:19:50.432 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 4, 2023-11-17T00:19:50.432 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 2944, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 448, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.002153046, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000061443, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000729322, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:50.433 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3, 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.001025073, 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000341691 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:50.434 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 448 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 1, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 102, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 160, 2023-11-17T00:19:50.435 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 169728, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.005554431, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.003608615, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.001003990, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:50.436 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 100, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.024659740, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000246597 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.437 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 169728 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 134, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 133, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 176096, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 75392, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:50.438 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.017698727, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.006364026, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.007471362, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.003462109, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 132, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.009209943, 2023-11-17T00:19:50.439 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000069772 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 176096, 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 75392 2023-11-17T00:19:50.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:50.441 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:50.442 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:50.443 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:50.444 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:50.444 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:50.444 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:50.444 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:50.445 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148823, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 27, 2023-11-17T00:19:50.446 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:50.447 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:50.448 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2023-11-17T00:19:50.449 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:50.450 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 135, 2023-11-17T00:19:50.450 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 157169, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 135, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 135, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 157169, 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:50.451 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:50.452 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:50.452 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:50.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:50.452 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:50.452 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-11-17T00:19:50.834 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2023-11-17T00:19:50.836 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2023-11-17T00:19:50.836 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2023-11-17T00:19:50.836 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2023-11-17T00:19:50.836 DEBUG:tasks.check_counter:Getting stats from k 2023-11-17T00:19:50.837 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-11-17T00:19:51.104 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:50 smithi087 ceph-mon[131182]: pgmap v340: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:51.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:50 smithi186 ceph-mon[131487]: pgmap v340: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:51.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:50 smithi006 ceph-mon[97840]: pgmap v340: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-11-17T00:19:51.787 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-17T00:19:53.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:52 smithi087 ceph-mon[131182]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:53.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:52 smithi186 ceph-mon[131487]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:53.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:52 smithi006 ceph-mon[97840]: pgmap v341: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:53.241 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2023-11-17T00:19:53.241 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 241, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 230, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1562496, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 131152, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:53.242 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.044131672, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.012883379, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.032540178, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.006351150, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 229, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.022789723, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000099518 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.243 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 1562496, 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 131152 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.244 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 176, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 182576, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.009560235, 2023-11-17T00:19:53.245 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.006078567, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.001405494, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 173, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.045240094, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000261503 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:53.246 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 182576 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:53.247 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 4, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 4, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 2944, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 448, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.000698462, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000043244, 2023-11-17T00:19:53.248 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000642062, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000716124, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000238708 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:53.249 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 448 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2023-11-17T00:19:53.250 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:53.251 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:53.252 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:53.253 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:53.254 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148754, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 31, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:53.255 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:53.256 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:53.257 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2023-11-17T00:19:53.258 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 243, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1528461, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 243, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 243, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1528461, 2023-11-17T00:19:53.259 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout: } 2023-11-17T00:19:53.260 INFO:teuthology.orchestra.run.smithi087.stdout:} 2023-11-17T00:19:53.666 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-11-17T00:19:53.666 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-11-17T00:19:53.666 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-11-17T00:19:53.666 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-17T00:19:53.666 DEBUG:tasks.check_counter:Getting stats from c 2023-11-17T00:19:53.667 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-11-17T00:19:53.723 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:19:53 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:19:53.424+0000 7f9c049b1700 -1 calc_pg_upmaps abort due to max <= 0 2023-11-17T00:19:54.641 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:19:55.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:54 smithi087 ceph-mon[131182]: pgmap v342: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:55.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:54 smithi186 ceph-mon[131487]: pgmap v342: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:55.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:54 smithi006 ceph-mon[97840]: pgmap v342: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:19:56.269 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 243, 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 230, 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1564896, 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 130416, 2023-11-17T00:19:56.270 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046770415, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.013688597, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.034131923, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006657991, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 230, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.028604607, 2023-11-17T00:19:56.271 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000124367 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1564896, 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 130416 2023-11-17T00:19:56.272 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 174, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 182224, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:56.273 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010337155, 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006742509, 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001336119, 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 171, 2023-11-17T00:19:56.274 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.030807965, 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000180163 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.276 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 182224 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:56.277 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:56.278 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000429119, 2023-11-17T00:19:56.278 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-11-17T00:19:56.278 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000351826, 2023-11-17T00:19:56.278 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:56.278 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:56.282 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:56.282 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:56.282 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:56.282 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.282 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:56.283 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:56.283 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:56.283 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:56.286 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.286 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-17T00:19:56.287 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-11-17T00:19:56.287 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.287 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-11-17T00:19:56.287 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-11-17T00:19:56.288 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-11-17T00:19:56.288 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.289 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-11-17T00:19:56.289 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:56.289 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:56.289 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:56.290 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:56.290 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:56.290 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:56.292 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:56.292 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:56.293 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:56.293 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:56.293 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:56.294 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:56.296 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:56.296 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:56.297 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:56.297 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:56.297 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:56.298 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:56.298 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:56.298 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:56.298 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:56.299 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:56.299 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:56.299 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:56.299 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:56.299 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:56.300 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:56.300 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:56.300 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:56.300 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:56.300 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:56.301 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-11-17T00:19:56.301 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:56.301 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:56.301 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:56.302 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:56.302 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148754, 2023-11-17T00:19:56.302 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 31, 2023-11-17T00:19:56.302 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:56.303 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:56.303 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-11-17T00:19:56.303 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-11-17T00:19:56.304 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:56.304 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:56.304 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:56.304 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:56.305 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-11-17T00:19:56.306 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:56.306 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-11-17T00:19:56.306 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:56.306 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-11-17T00:19:56.306 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:56.307 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-11-17T00:19:56.307 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:56.307 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:56.307 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:19:56.308 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-11-17T00:19:56.308 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:19:56.308 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 241, 2023-11-17T00:19:56.309 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1528201, 2023-11-17T00:19:56.309 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:56.309 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 241, 2023-11-17T00:19:56.309 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:19:56.310 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:19:56.310 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 241, 2023-11-17T00:19:56.310 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1528201, 2023-11-17T00:19:56.310 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:19:56.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:56.311 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:56.311 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:56.311 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:19:56.326 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:19:56.326 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-11-17T00:19:56.669 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2023-11-17T00:19:56.669 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2023-11-17T00:19:56.669 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2023-11-17T00:19:56.669 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2023-11-17T00:19:56.669 DEBUG:tasks.check_counter:Getting stats from f 2023-11-17T00:19:56.670 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-11-17T00:19:57.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:56 smithi087 ceph-mon[131182]: pgmap v343: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:57.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:56 smithi186 ceph-mon[131487]: pgmap v343: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:57.225 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:56 smithi006 ceph-mon[97840]: pgmap v343: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:19:57.506 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:19:59.042 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-11-17T00:19:59.043 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:19:59.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2023-11-17T00:19:59.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 4, 2023-11-17T00:19:59.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2944, 2023-11-17T00:19:59.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 448, 2023-11-17T00:19:59.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-11-17T00:19:59.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-11-17T00:19:59.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001413344, 2023-11-17T00:19:59.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000035537, 2023-11-17T00:19:59.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000511445, 2023-11-17T00:19:59.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:59.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:59.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2023-11-17T00:19:59.046 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000778308, 2023-11-17T00:19:59.046 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000259436 2023-11-17T00:19:59.046 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:59.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:59.047 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:59.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:59.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-17T00:19:59.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-11-17T00:19:59.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.048 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 240, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 229, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1562224, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 130560, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046032980, 2023-11-17T00:19:59.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.013868335, 2023-11-17T00:19:59.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.033828891, 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006692447, 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 228, 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012662400, 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000055536 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:59.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:59.052 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:59.052 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:59.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.054 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1562224, 2023-11-17T00:19:59.054 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 130560 2023-11-17T00:19:59.054 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.054 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:19:59.055 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2023-11-17T00:19:59.055 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 174, 2023-11-17T00:19:59.055 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 182224, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010695555, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.007071854, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001338599, 2023-11-17T00:19:59.056 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 171, 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.027305951, 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000159683 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:59.057 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 182224 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-11-17T00:19:59.058 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:19:59.059 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:19:59.060 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:19:59.061 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:19:59.062 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148732, 2023-11-17T00:19:59.063 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 31, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:19:59.064 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-11-17T00:19:59.065 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:19:59.066 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 242, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1528331, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 242, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 242, 2023-11-17T00:19:59.067 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1528331, 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:19:59.068 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-11-17T00:19:59.069 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:19:58 smithi186 ceph-mon[131487]: pgmap v344: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:59.104 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:19:58 smithi087 ceph-mon[131182]: pgmap v344: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:59.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:19:58 smithi006 ceph-mon[97840]: pgmap v344: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2023-11-17T00:19:59.486 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2023-11-17T00:19:59.487 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2023-11-17T00:19:59.487 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2023-11-17T00:19:59.487 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2023-11-17T00:19:59.487 DEBUG:tasks.check_counter:Getting stats from i 2023-11-17T00:19:59.487 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-11-17T00:20:00.442 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:20:01.068 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:00 smithi186 ceph-mon[131487]: pgmap v345: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:20:01.068 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:00 smithi186 ceph-mon[131487]: overall HEALTH_OK 2023-11-17T00:20:01.229 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:00 smithi006 ceph-mon[97840]: pgmap v345: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:20:01.230 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:00 smithi006 ceph-mon[97840]: overall HEALTH_OK 2023-11-17T00:20:01.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:00 smithi087 ceph-mon[131182]: pgmap v345: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2023-11-17T00:20:01.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:00 smithi087 ceph-mon[131182]: overall HEALTH_OK 2023-11-17T00:20:02.022 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1524, 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1513, 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 238377755, 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 527518, 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2023-11-17T00:20:02.023 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.279294928, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.043587330, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.182882196, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.048914917, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1512, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.186764147, 2023-11-17T00:20:02.024 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000123521 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1585840, 2023-11-17T00:20:02.025 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 132544 2023-11-17T00:20:02.032 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.032 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 570, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 742, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 228161120, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 524490, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 7, 2023-11-17T00:20:02.033 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.152444789, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.019510104, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.111516656, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.015632774, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 739, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.330801994, 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000447634 2023-11-17T00:20:02.034 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 347776 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.035 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 931, 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 931, 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 277148012, 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 288752, 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2023-11-17T00:20:02.036 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.208474219, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.022434298, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.152040756, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.027783297, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 930, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.149020958, 2023-11-17T00:20:02.037 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000160237 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-17T00:20:02.038 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-11-17T00:20:02.039 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-11-17T00:20:02.040 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3878, 2023-11-17T00:20:02.040 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.918411014, 2023-11-17T00:20:02.040 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000494690 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.041 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.042 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2023-11-17T00:20:02.043 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 187089, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 130947, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 56142, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 21, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 299, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-11-17T00:20:02.044 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 2, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-11-17T00:20:02.045 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-11-17T00:20:02.046 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 246, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 7, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-11-17T00:20:02.047 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-11-17T00:20:02.048 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2023-11-17T00:20:02.049 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2023-11-17T00:20:02.050 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-11-17T00:20:02.051 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2023-11-17T00:20:02.052 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2023-11-17T00:20:02.053 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 760544883, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 0, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 760652289, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.054 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 238124 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 87372, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 137378, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 50006, 2023-11-17T00:20:02.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 2581, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 2969, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 388, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 187089, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 187392, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 303, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2023-11-17T00:20:02.056 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 453196, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207156 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2023-11-17T00:20:02.057 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.058 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.059 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.060 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.061 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.062 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.063 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.064 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.065 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-11-17T00:20:02.066 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.067 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-11-17T00:20:02.068 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.068 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.068 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.068 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.068 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-11-17T00:20:02.069 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.069 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.069 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.069 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.069 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.070 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-11-17T00:20:02.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.072 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-11-17T00:20:02.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.074 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.076 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-11-17T00:20:02.077 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 2, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 0, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 2604 2023-11-17T00:20:02.078 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:20:02.079 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:20:02.080 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:20:02.081 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-11-17T00:20:02.082 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 3209858, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 87473, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:20:02.083 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-11-17T00:20:02.084 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 9880, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 192, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 331920811, 2023-11-17T00:20:02.085 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2771434, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-11-17T00:20:02.086 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2765, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 26, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 4, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2765, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2765, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 19.384587889, 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.007010700 2023-11-17T00:20:02.087 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2765, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2766 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2765, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2765, 2023-11-17T00:20:02.088 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1879, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 883, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-11-17T00:20:02.089 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 2, 2023-11-17T00:20:02.090 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-11-17T00:20:02.091 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-11-17T00:20:02.092 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-11-17T00:20:02.093 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 102, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 9, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 11, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 11, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-11-17T00:20:02.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-11-17T00:20:02.095 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2023-11-17T00:20:02.096 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-11-17T00:20:02.097 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 3021, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 743398981, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 3021, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 3021, 2023-11-17T00:20:02.098 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 743398981, 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:02.099 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2765, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 741339230, 2023-11-17T00:20:02.100 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2765, 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 741339230, 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.101 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2765, 2023-11-17T00:20:02.102 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2765, 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2765, 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2765, 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.103 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:02.104 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.105 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x5588eab18000": { 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-11-17T00:20:02.106 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:02.107 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:02.108 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.108 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:02.108 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-11-17T00:20:02.434 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2023-11-17T00:20:02.434 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-11-17T00:20:02.434 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-11-17T00:20:02.434 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2023-11-17T00:20:02.434 DEBUG:tasks.check_counter:Getting stats from l 2023-11-17T00:20:02.435 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-11-17T00:20:03.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:02 smithi006 ceph-mon[97840]: pgmap v346: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:20:03.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:02 smithi087 ceph-mon[131182]: pgmap v346: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:20:03.366 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-17T00:20:03.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:02 smithi186 ceph-mon[131487]: pgmap v346: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-11-17T00:20:04.911 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-11-17T00:20:04.912 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-11-17T00:20:04.912 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2023-11-17T00:20:04.912 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 4, 2023-11-17T00:20:04.912 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2944, 2023-11-17T00:20:04.912 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 448, 2023-11-17T00:20:04.913 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-11-17T00:20:04.913 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-11-17T00:20:04.913 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001727123, 2023-11-17T00:20:04.913 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000054522, 2023-11-17T00:20:04.914 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000793972, 2023-11-17T00:20:04.914 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:20:04.914 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:04.914 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2023-11-17T00:20:04.915 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000823951, 2023-11-17T00:20:04.915 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000274650 2023-11-17T00:20:04.915 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.915 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:04.916 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:04.916 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:04.916 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:04.916 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.917 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-11-17T00:20:04.917 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-11-17T00:20:04.917 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.917 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-11-17T00:20:04.918 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 245, 2023-11-17T00:20:04.918 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 234, 2023-11-17T00:20:04.918 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1563584, 2023-11-17T00:20:04.918 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 133520, 2023-11-17T00:20:04.919 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-11-17T00:20:04.919 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-11-17T00:20:04.919 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.046612850, 2023-11-17T00:20:04.919 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.013558446, 2023-11-17T00:20:04.920 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.035017331, 2023-11-17T00:20:04.920 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.006862348, 2023-11-17T00:20:04.920 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:04.920 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 233, 2023-11-17T00:20:04.921 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.018643155, 2023-11-17T00:20:04.921 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000080013 2023-11-17T00:20:04.921 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.921 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:04.922 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:04.922 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:04.922 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:04.922 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.923 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1563584, 2023-11-17T00:20:04.923 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 133520 2023-11-17T00:20:04.923 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.923 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-11-17T00:20:04.924 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2023-11-17T00:20:04.924 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 179, 2023-11-17T00:20:04.924 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 320, 2023-11-17T00:20:04.924 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 183104, 2023-11-17T00:20:04.925 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2023-11-17T00:20:04.925 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2023-11-17T00:20:04.925 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.010616611, 2023-11-17T00:20:04.925 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006813661, 2023-11-17T00:20:04.926 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001572260, 2023-11-17T00:20:04.926 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-17T00:20:04.926 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-11-17T00:20:04.926 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 176, 2023-11-17T00:20:04.926 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.039334983, 2023-11-17T00:20:04.927 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000223494 2023-11-17T00:20:04.927 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.927 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-11-17T00:20:04.927 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:04.928 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:04.928 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:04.928 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.928 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-17T00:20:04.929 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 183104 2023-11-17T00:20:04.929 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.929 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-11-17T00:20:04.929 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-11-17T00:20:04.930 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-11-17T00:20:04.930 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.930 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-11-17T00:20:04.930 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-11-17T00:20:04.931 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-11-17T00:20:04.931 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-11-17T00:20:04.931 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-11-17T00:20:04.931 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-11-17T00:20:04.932 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-11-17T00:20:04.933 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-17T00:20:04.933 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-11-17T00:20:04.934 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-17T00:20:04.934 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-11-17T00:20:04.934 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-11-17T00:20:04.935 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-11-17T00:20:04.935 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-17T00:20:04.935 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-11-17T00:20:04.935 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-11-17T00:20:04.935 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-11-17T00:20:04.936 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-11-17T00:20:04.936 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-11-17T00:20:04.936 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-17T00:20:04.936 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-11-17T00:20:04.937 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-17T00:20:04.937 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-11-17T00:20:04.937 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-11-17T00:20:04.937 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-11-17T00:20:04.938 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-11-17T00:20:04.938 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-11-17T00:20:04.938 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-17T00:20:04.939 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-11-17T00:20:04.939 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-11-17T00:20:04.939 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-11-17T00:20:04.940 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-11-17T00:20:04.940 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-11-17T00:20:04.940 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-11-17T00:20:04.940 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-11-17T00:20:04.941 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-11-17T00:20:04.941 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-11-17T00:20:04.941 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 149745, 2023-11-17T00:20:04.942 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 31, 2023-11-17T00:20:04.942 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-11-17T00:20:04.942 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-11-17T00:20:04.942 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-11-17T00:20:04.943 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-11-17T00:20:04.943 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-11-17T00:20:04.943 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-11-17T00:20:04.943 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-11-17T00:20:04.944 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-11-17T00:20:04.944 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-11-17T00:20:04.944 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-11-17T00:20:04.944 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-11-17T00:20:04.945 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-11-17T00:20:04.946 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-11-17T00:20:04.946 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-11-17T00:20:04.946 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-11-17T00:20:04.946 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-11-17T00:20:04.947 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-11-17T00:20:04.947 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-11-17T00:20:04.947 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-11-17T00:20:04.947 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-11-17T00:20:04.948 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-11-17T00:20:04.948 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-17T00:20:04.948 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-11-17T00:20:04.948 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-11-17T00:20:04.949 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-11-17T00:20:04.949 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 247, 2023-11-17T00:20:04.949 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1528981, 2023-11-17T00:20:04.949 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-11-17T00:20:04.950 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 247, 2023-11-17T00:20:04.950 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-11-17T00:20:04.950 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-11-17T00:20:04.950 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 247, 2023-11-17T00:20:04.951 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1528981, 2023-11-17T00:20:04.951 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-11-17T00:20:04.951 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-11-17T00:20:04.951 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-11-17T00:20:04.952 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-11-17T00:20:04.952 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:04.952 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-11-17T00:20:04.952 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-11-17T00:20:05.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:04 smithi006 ceph-mon[97840]: pgmap v347: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-17T00:20:05.335 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:04 smithi186 ceph-mon[131487]: pgmap v347: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-17T00:20:05.352 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-11-17T00:20:05.352 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-11-17T00:20:05.352 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-11-17T00:20:05.352 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-17T00:20:05.353 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-17T00:20:05.442 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-17T00:20:05.443 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:20:05.443 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-17T00:20:05.444 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:04 smithi087 ceph-mon[131182]: pgmap v347: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-11-17T00:20:05.547 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-17T00:20:05.547 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-17T00:20:05.643 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:20:06.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.012+0000 7f70013ff700 1 -- 172.21.15.6:0/2818788865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc101390 msgr2=0x7f6ffc101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.012+0000 7f70013ff700 1 --2- 172.21.15.6:0/2818788865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc101390 0x7f6ffc101760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00098f0 tx=0x7f6ff0009c00 comp rx=0 tx=0).stop 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.013+0000 7f70013ff700 1 -- 172.21.15.6:0/2818788865 shutdown_connections 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.013+0000 7f70013ff700 1 --2- 172.21.15.6:0/2818788865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.013+0000 7f70013ff700 1 --2- 172.21.15.6:0/2818788865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc101390 0x7f6ffc101760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.013+0000 7f70013ff700 1 --2- 172.21.15.6:0/2818788865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.013+0000 7f70013ff700 1 -- 172.21.15.6:0/2818788865 >> 172.21.15.6:0/2818788865 conn(0x7f6ffc077060 msgr2=0x7f6ffc077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.014+0000 7f70013ff700 1 -- 172.21.15.6:0/2818788865 shutdown_connections 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.014+0000 7f70013ff700 1 -- 172.21.15.6:0/2818788865 wait complete. 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.016+0000 7f70013ff700 1 Processor -- start 2023-11-17T00:20:06.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.016+0000 7f70013ff700 1 -- start start 2023-11-17T00:20:06.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ffc101390 0x7f6ffc109b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc106c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1073a0 con 0x7f6ffc101390 2023-11-17T00:20:06.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc107510 con 0x7f6ffc10a8b0 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f70013ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1076d0 con 0x7f6ffc101ca0 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f6ffa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f6ffb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f6ffa7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:45620/0 (socket says 172.21.15.6:45620) 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.017+0000 7f6ffa7fc700 1 -- 172.21.15.6:0/2582259098 learned_addr learned my addr 172.21.15.6:0/2582259098 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.018+0000 7f6ffa7fc700 1 -- 172.21.15.6:0/2582259098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 msgr2=0x7f6ffc106c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.018+0000 7f6ffa7fc700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc106c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.018+0000 7f6ffa7fc700 1 -- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ffc101390 msgr2=0x7f6ffc109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.018+0000 7f6ffa7fc700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ffc101390 0x7f6ffc109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.018+0000 7f6ffa7fc700 1 -- 172.21.15.6:0/2582259098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009520 con 0x7f6ffc101ca0 2023-11-17T00:20:06.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f6ffa7fc700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00d950 tx=0x7f6fec00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:06.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f6ffb7fe700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-17T00:20:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec009940 con 0x7f6ffc101ca0 2023-11-17T00:20:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fec010460 con 0x7f6ffc101ca0 2023-11-17T00:20:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc100800 con 0x7f6ffc101ca0 2023-11-17T00:20:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00f620 con 0x7f6ffc101ca0 2023-11-17T00:20:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.019+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc100670 con 0x7f6ffc101ca0 2023-11-17T00:20:06.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.021+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc100990 con 0x7f6ffc101ca0 2023-11-17T00:20:06.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.024+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6fec0108d0 con 0x7f6ffc101ca0 2023-11-17T00:20:06.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.026+0000 7f6fe3fff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 0x7f6fe40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.026+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f6fec0ccda0 con 0x7f6ffc101ca0 2023-11-17T00:20:06.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.026+0000 7f6ffaffd700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 0x7f6fe40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.028+0000 7f6ffaffd700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 0x7f6fe40a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c010 tx=0x7f6ff0009f50 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:06.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.032+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6fec095d50 con 0x7f6ffc101ca0 2023-11-17T00:20:06.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.443+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ffc108130 con 0x7f6ffc101ca0 2023-11-17T00:20:06.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.447+0000 7f6fe3fff700 1 -- 172.21.15.6:0/2582259098 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+25168 (secure 0 0 0) 0x7f6fec016090 con 0x7f6ffc101ca0 2023-11-17T00:20:06.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:20:06.454 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":102,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:19:08.611438+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6800/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1767647821":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/2273227531":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/4272143857":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3662483346":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:20:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 msgr2=0x7f6fe40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 0x7f6fe40a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c010 tx=0x7f6ff0009f50 comp rx=0 tx=0).stop 2023-11-17T00:20:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 msgr2=0x7f6ffc10a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00d950 tx=0x7f6fec00dc60 comp rx=0 tx=0).stop 2023-11-17T00:20:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 shutdown_connections 2023-11-17T00:20:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6fe40a1340 0x7f6fe40a37f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ffc101390 0x7f6ffc109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ffc10a8b0 0x7f6ffc106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 --2- 172.21.15.6:0/2582259098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc101ca0 0x7f6ffc10a080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 >> 172.21.15.6:0/2582259098 conn(0x7f6ffc077060 msgr2=0x7f6ffc10ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.452+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 shutdown_connections 2023-11-17T00:20:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.453+0000 7f70013ff700 1 -- 172.21.15.6:0/2582259098 wait complete. 2023-11-17T00:20:06.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-17T00:01:28.259100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-17T00:06:11.024961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-17T00:06:12.046799+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-17T00:06:15.576749+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-17T00:20:06.490 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-17T00:20:06.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 -- 172.21.15.6:0/2157452389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838108400 msgr2=0x7f0838112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 --2- 172.21.15.6:0/2157452389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838108400 0x7f0838112e70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0828009990 tx=0x7f0828009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 -- 172.21.15.6:0/2157452389 shutdown_connections 2023-11-17T00:20:06.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 --2- 172.21.15.6:0/2157452389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08381133b0 0x7f0838115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 --2- 172.21.15.6:0/2157452389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838107af0 0x7f0838107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 --2- 172.21.15.6:0/2157452389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838108400 0x7f0838112e70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 -- 172.21.15.6:0/2157452389 >> 172.21.15.6:0/2157452389 conn(0x7f083806c740 msgr2=0x7f0838071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 -- 172.21.15.6:0/2157452389 shutdown_connections 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.861+0000 7f08407ff700 1 -- 172.21.15.6:0/2157452389 wait complete. 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.863+0000 7f08407ff700 1 Processor -- start 2023-11-17T00:20:06.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.863+0000 7f08407ff700 1 -- start start 2023-11-17T00:20:06.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 0x7f083810e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 0x7f0838109160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838109930 con 0x7f0838107af0 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838109aa0 con 0x7f0838108400 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f08407ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838109c60 con 0x7f08381133b0 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.864+0000 7f083dd9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083dd9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59684/0 (socket says 172.21.15.6:59684) 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083dd9a700 1 -- 172.21.15.6:0/2098429168 learned_addr learned my addr 172.21.15.6:0/2098429168 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083e59b700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 0x7f083810e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083dd9a700 1 -- 172.21.15.6:0/2098429168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 msgr2=0x7f0838109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083ed9c700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 0x7f0838109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.865+0000 7f083dd9a700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 0x7f0838109160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083dd9a700 1 -- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 msgr2=0x7f083810e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083dd9a700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 0x7f083810e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083dd9a700 1 -- 172.21.15.6:0/2098429168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280095f0 con 0x7f0838108400 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083ed9c700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 0x7f0838109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083e59b700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 0x7f083810e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-17T00:20:06.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.866+0000 7f083dd9a700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08280095c0 tx=0x7f082800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:06.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.867+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082801d070 con 0x7f0838108400 2023-11-17T00:20:06.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.867+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0828005510 con 0x7f0838108400 2023-11-17T00:20:06.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.867+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828004bd0 con 0x7f0838108400 2023-11-17T00:20:06.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.867+0000 7f08407ff700 1 -- 172.21.15.6:0/2098429168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0838118c50 con 0x7f0838108400 2023-11-17T00:20:06.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.867+0000 7f08407ff700 1 -- 172.21.15.6:0/2098429168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0838118ac0 con 0x7f0838108400 2023-11-17T00:20:06.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.869+0000 7f08407ff700 1 -- 172.21.15.6:0/2098429168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0838118de0 con 0x7f0838108400 2023-11-17T00:20:06.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.872+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0828004d30 con 0x7f0838108400 2023-11-17T00:20:06.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.875+0000 7f082f7fe700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 0x7f08240a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:06.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.875+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f08280ce020 con 0x7f0838108400 2023-11-17T00:20:06.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.876+0000 7f083e59b700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 0x7f08240a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:06.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.877+0000 7f083e59b700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 0x7f08240a3870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0834005fd0 tx=0x7f0834005ee0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:06.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:06.880+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0828097060 con 0x7f0838108400 2023-11-17T00:20:07.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:06 smithi006 ceph-mon[97840]: pgmap v348: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-11-17T00:20:07.226 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:06 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2582259098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:07.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.286+0000 7f08407ff700 1 -- 172.21.15.6:0/2098429168 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0838112f90 con 0x7f0838108400 2023-11-17T00:20:07.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.287+0000 7f082f7fe700 1 -- 172.21.15.6:0/2098429168 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f082809a680 con 0x7f0838108400 2023-11-17T00:20:07.289 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-17T00:20:07.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.291+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 msgr2=0x7f08240a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:07.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.291+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 0x7f08240a3870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0834005fd0 tx=0x7f0834005ee0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 msgr2=0x7f083810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:07.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08280095c0 tx=0x7f082800f690 comp rx=0 tx=0).stop 2023-11-17T00:20:07.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 shutdown_connections 2023-11-17T00:20:07.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f08240a13c0 0x7f08240a3870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0838107af0 0x7f083810e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08381133b0 0x7f0838109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 --2- 172.21.15.6:0/2098429168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0838108400 0x7f083810e600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 >> 172.21.15.6:0/2098429168 conn(0x7f083806c740 msgr2=0x7f083819e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:07.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.292+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 shutdown_connections 2023-11-17T00:20:07.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.293+0000 7f082d7fa700 1 -- 172.21.15.6:0/2098429168 wait complete. 2023-11-17T00:20:07.333 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-17T00:20:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:06 smithi087 ceph-mon[131182]: pgmap v348: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-11-17T00:20:07.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:06 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2582259098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:07.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:06 smithi186 ceph-mon[131487]: pgmap v348: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-11-17T00:20:07.369 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:06 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2582259098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:07.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 -- 172.21.15.6:0/3527571868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8067dc0 msgr2=0x7f05a8068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:07.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 --2- 172.21.15.6:0/3527571868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8067dc0 0x7f05a8068230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0598009990 tx=0x7f0598009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 -- 172.21.15.6:0/3527571868 shutdown_connections 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 --2- 172.21.15.6:0/3527571868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05a8105250 0x7f05a8105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 --2- 172.21.15.6:0/3527571868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8068770 0x7f05a810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 --2- 172.21.15.6:0/3527571868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8067dc0 0x7f05a8068230 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.711+0000 7f05b03ff700 1 -- 172.21.15.6:0/3527571868 >> 172.21.15.6:0/3527571868 conn(0x7f05a80763a0 msgr2=0x7f05a80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.712+0000 7f05b03ff700 1 -- 172.21.15.6:0/3527571868 shutdown_connections 2023-11-17T00:20:07.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.712+0000 7f05b03ff700 1 -- 172.21.15.6:0/3527571868 wait complete. 2023-11-17T00:20:07.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.713+0000 7f05b03ff700 1 Processor -- start 2023-11-17T00:20:07.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 -- start start 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8067dc0 0x7f05a8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05a8068770 0x7f05a8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819c580 con 0x7f05a8068770 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819c6f0 con 0x7f05a8105250 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.714+0000 7f05b03ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a819c8b0 con 0x7f05a8067dc0 2023-11-17T00:20:07.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:07.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59710/0 (socket says 172.21.15.6:59710) 2023-11-17T00:20:07.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 -- 172.21.15.6:0/1282129276 learned_addr learned my addr 172.21.15.6:0/1282129276 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:07.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae19b700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8067dc0 0x7f05a8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:07.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 -- 172.21.15.6:0/1282129276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8067dc0 msgr2=0x7f05a8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:07.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8067dc0 0x7f05a8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 -- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05a8068770 msgr2=0x7f05a8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:07.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05a8068770 0x7f05a8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:07.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 -- 172.21.15.6:0/1282129276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980095f0 con 0x7f05a8105250 2023-11-17T00:20:07.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.715+0000 7f05ae99c700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05a000ef90 tx=0x7f05a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:07.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.716+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0018070 con 0x7f05a8105250 2023-11-17T00:20:07.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.716+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a000f040 con 0x7f05a8105250 2023-11-17T00:20:07.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.716+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a8110420 con 0x7f05a8105250 2023-11-17T00:20:07.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.716+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8110290 con 0x7f05a8105250 2023-11-17T00:20:07.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.717+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0003dc0 con 0x7f05a8105250 2023-11-17T00:20:07.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.719+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c005310 con 0x7f05a8105250 2023-11-17T00:20:07.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.724+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f05a0004020 con 0x7f05a8105250 2023-11-17T00:20:07.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.729+0000 7f059f7fe700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 0x7f05940a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:07.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.729+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f05a00cccc0 con 0x7f05a8105250 2023-11-17T00:20:07.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.731+0000 7f05ae19b700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 0x7f05940a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:07.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.731+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f05a0095d00 con 0x7f05a8105250 2023-11-17T00:20:07.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:07.733+0000 7f05ae19b700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 0x7f05940a3870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005950 tx=0x7f05a400b410 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:08.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.131+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f058c005f60 con 0x7f05a8105250 2023-11-17T00:20:08.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.132+0000 7f059f7fe700 1 -- 172.21.15.6:0/1282129276 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05a0099320 con 0x7f05a8105250 2023-11-17T00:20:08.139 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:20:08.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 msgr2=0x7f05940a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:08.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 0x7f05940a3870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05a4005950 tx=0x7f05a400b410 comp rx=0 tx=0).stop 2023-11-17T00:20:08.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 msgr2=0x7f05a819bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:08.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05a000ef90 tx=0x7f05a000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 shutdown_connections 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f05940a13c0 0x7f05940a3870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a8105250 0x7f05a819bf80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05a8067dc0 0x7f05a8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 --2- 172.21.15.6:0/1282129276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05a8068770 0x7f05a8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 >> 172.21.15.6:0/1282129276 conn(0x7f05a80763a0 msgr2=0x7f05a810c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 shutdown_connections 2023-11-17T00:20:08.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.137+0000 7f05b03ff700 1 -- 172.21.15.6:0/1282129276 wait complete. 2023-11-17T00:20:08.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:07 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2098429168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:20:08.174 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-17T00:20:08.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:07 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2098429168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:20:08.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:07 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2098429168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-17T00:20:08.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.553+0000 7f889a5ff700 1 -- 172.21.15.6:0/2533986581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 msgr2=0x7f8894105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:08.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.553+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2533986581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 0x7f8894105620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b0a0 tx=0x7f888c00b3b0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 -- 172.21.15.6:0/2533986581 shutdown_connections 2023-11-17T00:20:08.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2533986581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f889410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2533986581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 0x7f8894105620 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2533986581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 0x7f8894068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 -- 172.21.15.6:0/2533986581 >> 172.21.15.6:0/2533986581 conn(0x7f88940763a0 msgr2=0x7f88940767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:08.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 -- 172.21.15.6:0/2533986581 shutdown_connections 2023-11-17T00:20:08.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.554+0000 7f889a5ff700 1 -- 172.21.15.6:0/2533986581 wait complete. 2023-11-17T00:20:08.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.556+0000 7f889a5ff700 1 Processor -- start 2023-11-17T00:20:08.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.556+0000 7f889a5ff700 1 -- start start 2023-11-17T00:20:08.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 0x7f88941986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:08.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:08.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 0x7f889419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:08.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419c590 con 0x7f8894067dc0 2023-11-17T00:20:08.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419c700 con 0x7f8894068770 2023-11-17T00:20:08.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f889a5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419c8c0 con 0x7f8894105250 2023-11-17T00:20:08.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f8893fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 0x7f88941986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:08.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f88937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:08.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f88937fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:59734/0 (socket says 172.21.15.6:59734) 2023-11-17T00:20:08.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.557+0000 7f88937fe700 1 -- 172.21.15.6:0/2441130730 learned_addr learned my addr 172.21.15.6:0/2441130730 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:08.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.558+0000 7f88937fe700 1 -- 172.21.15.6:0/2441130730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 msgr2=0x7f889419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:08.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.558+0000 7f88937fe700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 0x7f889419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.558+0000 7f88937fe700 1 -- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 msgr2=0x7f88941986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:08.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.558+0000 7f88937fe700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 0x7f88941986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:08.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.558+0000 7f88937fe700 1 -- 172.21.15.6:0/2441130730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c009d00 con 0x7f8894068770 2023-11-17T00:20:08.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.560+0000 7f88937fe700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f888400b550 tx=0x7f888400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:08.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.561+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884011830 con 0x7f8894068770 2023-11-17T00:20:08.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.561+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884011e50 con 0x7f8894068770 2023-11-17T00:20:08.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.561+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888400f2e0 con 0x7f8894068770 2023-11-17T00:20:08.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.562+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88941104f0 con 0x7f8894068770 2023-11-17T00:20:08.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.562+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894110360 con 0x7f8894068770 2023-11-17T00:20:08.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.564+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894110680 con 0x7f8894068770 2023-11-17T00:20:08.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.571+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8884011990 con 0x7f8894068770 2023-11-17T00:20:08.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.573+0000 7f88917fa700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 0x7f887c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:08.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.573+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f88840ccad0 con 0x7f8894068770 2023-11-17T00:20:08.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.573+0000 7f8893fff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 0x7f887c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:08.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.574+0000 7f8893fff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 0x7f887c0a38b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b780 tx=0x7f888c005040 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:08.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.577+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8884095b90 con 0x7f8894068770 2023-11-17T00:20:08.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.991+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88941a57a0 con 0x7f8894068770 2023-11-17T00:20:08.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.993+0000 7f88917fa700 1 -- 172.21.15.6:0/2441130730 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88840991b0 con 0x7f8894068770 2023-11-17T00:20:08.995 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-17T00:20:09.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 msgr2=0x7f887c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:09.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 0x7f887c0a38b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b780 tx=0x7f888c005040 comp rx=0 tx=0).stop 2023-11-17T00:20:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 msgr2=0x7f8894198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f888400b550 tx=0x7f888400b860 comp rx=0 tx=0).stop 2023-11-17T00:20:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 shutdown_connections 2023-11-17T00:20:09.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f887c0a1400 0x7f887c0a38b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8894067dc0 0x7f88941986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8894105250 0x7f889419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 --2- 172.21.15.6:0/2441130730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8894068770 0x7f8894198be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:08.999+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 >> 172.21.15.6:0/2441130730 conn(0x7f88940763a0 msgr2=0x7f88940fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.000+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 shutdown_connections 2023-11-17T00:20:09.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.000+0000 7f889a5ff700 1 -- 172.21.15.6:0/2441130730 wait complete. 2023-11-17T00:20:09.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-17T00:20:09.165 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:08 smithi006 ceph-mon[97840]: pgmap v349: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:09.166 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:08 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1282129276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:20:09.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:08 smithi087 ceph-mon[131182]: pgmap v349: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:09.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:08 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1282129276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:20:09.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:08 smithi186 ceph-mon[131487]: pgmap v349: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:09.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:08 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1282129276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-17T00:20:09.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 -- 172.21.15.6:0/3570930294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 msgr2=0x7f873c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:09.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 --2- 172.21.15.6:0/3570930294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 0x7f873c0ffee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f87300098f0 tx=0x7f8730009c00 comp rx=0 tx=0).stop 2023-11-17T00:20:09.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 -- 172.21.15.6:0/3570930294 shutdown_connections 2023-11-17T00:20:09.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 --2- 172.21.15.6:0/3570930294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f873c100420 0x7f873c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 --2- 172.21.15.6:0/3570930294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c105250 0x7f873c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 --2- 172.21.15.6:0/3570930294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 0x7f873c0ffee0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.416+0000 7f87415ff700 1 -- 172.21.15.6:0/3570930294 >> 172.21.15.6:0/3570930294 conn(0x7f873c0763a0 msgr2=0x7f873c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:09.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.417+0000 7f87415ff700 1 -- 172.21.15.6:0/3570930294 shutdown_connections 2023-11-17T00:20:09.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.417+0000 7f87415ff700 1 -- 172.21.15.6:0/3570930294 wait complete. 2023-11-17T00:20:09.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.418+0000 7f87415ff700 1 Processor -- start 2023-11-17T00:20:09.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.419+0000 7f87415ff700 1 -- start start 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 0x7f873c105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f873c105250 0x7f873c106a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1a8d60 con 0x7f873c0ffa70 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c198430 con 0x7f873c105250 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f87415ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1985f0 con 0x7f873c100420 2023-11-17T00:20:09.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f873a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f873a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37346/0 (socket says 172.21.15.6:37346) 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.420+0000 7f873a7fc700 1 -- 172.21.15.6:0/1766657667 learned_addr learned my addr 172.21.15.6:0/1766657667 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 -- 172.21.15.6:0/1766657667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f873c105250 msgr2=0x7f873c106a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f873c105250 0x7f873c106a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 -- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 msgr2=0x7f873c105fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:20:09.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 0x7f873c105fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 -- 172.21.15.6:0/1766657667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009520 con 0x7f873c100420 2023-11-17T00:20:09.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.421+0000 7f873a7fc700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8730009230 tx=0x7f873000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:09.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.422+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873001c070 con 0x7f873c100420 2023-11-17T00:20:09.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.422+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8730005040 con 0x7f873c100420 2023-11-17T00:20:09.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.422+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730003a20 con 0x7f873c100420 2023-11-17T00:20:09.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.422+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c10f2d0 con 0x7f873c100420 2023-11-17T00:20:09.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.423+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c10f140 con 0x7f873c100420 2023-11-17T00:20:09.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.424+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c10f460 con 0x7f873c100420 2023-11-17T00:20:09.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.427+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8730020030 con 0x7f873c100420 2023-11-17T00:20:09.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.430+0000 7f8723fff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 0x7f87240c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:09.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.430+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f87300ce090 con 0x7f873c100420 2023-11-17T00:20:09.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.431+0000 7f873affd700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 0x7f87240c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:09.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.432+0000 7f873affd700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 0x7f87240c5910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f872c009730 tx=0x7f872c006cb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:09.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.438+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8730097130 con 0x7f873c100420 2023-11-17T00:20:09.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.840+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f873c198f70 con 0x7f873c100420 2023-11-17T00:20:09.845 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-17T00:20:09.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.841+0000 7f8723fff700 1 -- 172.21.15.6:0/1766657667 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) v1 ==== 103+0+11 (secure 0 0 0) 0x7f873009a750 con 0x7f873c100420 2023-11-17T00:20:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.845+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 msgr2=0x7f87240c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.845+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 0x7f87240c5910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f872c009730 tx=0x7f872c006cb0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.845+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 msgr2=0x7f873c106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.845+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8730009230 tx=0x7f873000b610 comp rx=0 tx=0).stop 2023-11-17T00:20:09.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 shutdown_connections 2023-11-17T00:20:09.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f87240c3460 0x7f87240c5910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f873c0ffa70 0x7f873c105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f873c105250 0x7f873c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 --2- 172.21.15.6:0/1766657667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f873c100420 0x7f873c106520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:09.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.846+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 >> 172.21.15.6:0/1766657667 conn(0x7f873c0763a0 msgr2=0x7f873c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:09.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.848+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 shutdown_connections 2023-11-17T00:20:09.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:09.848+0000 7f87415ff700 1 -- 172.21.15.6:0/1766657667 wait complete. 2023-11-17T00:20:09.904 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-17T00:20:10.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2441130730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:09 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1766657667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 -- 172.21.15.6:0/71225365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 msgr2=0x7fcb2c115890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 --2- 172.21.15.6:0/71225365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c115890 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009990 tx=0x7fcb28009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 -- 172.21.15.6:0/71225365 shutdown_connections 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 --2- 172.21.15.6:0/71225365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c115890 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 --2- 172.21.15.6:0/71225365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2c10ea60 0x7fcb2c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 --2- 172.21.15.6:0/71225365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c108b00 0x7fcb2c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.268+0000 7fcb345ff700 1 -- 172.21.15.6:0/71225365 >> 172.21.15.6:0/71225365 conn(0x7fcb2c072910 msgr2=0x7fcb2c06f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:10.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.269+0000 7fcb345ff700 1 -- 172.21.15.6:0/71225365 shutdown_connections 2023-11-17T00:20:10.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.269+0000 7fcb345ff700 1 -- 172.21.15.6:0/71225365 wait complete. 2023-11-17T00:20:10.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.271+0000 7fcb345ff700 1 Processor -- start 2023-11-17T00:20:10.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.271+0000 7fcb345ff700 1 -- start start 2023-11-17T00:20:10.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb345ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2c108b00 0x7fcb2c1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb345ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb345ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c10ea60 0x7fcb2c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb31b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb31b9a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40260/0 (socket says 172.21.15.6:40260) 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb31b9a700 1 -- 172.21.15.6:0/2391606422 learned_addr learned my addr 172.21.15.6:0/2391606422 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c1a11a0 con 0x7fcb2c10ea60 2023-11-17T00:20:10.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.272+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c1a1310 con 0x7fcb2c1094b0 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c1a14d0 con 0x7fcb2c108b00 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb32b9c700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c10ea60 0x7fcb2c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb31b9a700 1 -- 172.21.15.6:0/2391606422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2c108b00 msgr2=0x7fcb2c1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb31b9a700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2c108b00 0x7fcb2c1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb31b9a700 1 -- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c10ea60 msgr2=0x7fcb2c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:10.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb31b9a700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c10ea60 0x7fcb2c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.273+0000 7fcb31b9a700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280095f0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.274+0000 7fcb31b9a700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000ef10 tx=0x7fcb2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.274+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000cad0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.275+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2000f040 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.275+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb200146e0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.275+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c118b40 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.275+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1189b0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.276+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c118cd0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.281+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcb20007500 con 0x7fcb2c1094b0 2023-11-17T00:20:10.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.283+0000 7fcb1f7fe700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 0x7fcb180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:10.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.283+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fcb200cd520 con 0x7fcb2c1094b0 2023-11-17T00:20:10.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.284+0000 7fcb3239b700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 0x7fcb180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:10.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.285+0000 7fcb3239b700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 0x7fcb180a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c109b90 tx=0x7fcb24009470 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:10.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.289+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb20096530 con 0x7fcb2c1094b0 2023-11-17T00:20:10.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:09 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2441130730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:09 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1766657667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:09 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2441130730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:09 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1766657667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-17T00:20:10.729 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/879660796 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.730 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/879660796 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.730 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1767647821 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.730 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2273227531 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.730 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/1821215069 2023-11-17T23:59:01.653365+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4157530954 2023-11-17T23:59:01.653365+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6800/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/4272143857 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3407461840 2023-11-17T23:58:33.059992+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3069406217 2023-11-17T23:58:33.059992+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3662483346 2023-11-18T00:08:53.335648+0000 2023-11-17T00:20:10.731 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/3715783566 2023-11-17T23:58:33.059992+0000 2023-11-17T00:20:10.732 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:6801/610057204 2023-11-17T23:59:01.653365+0000 2023-11-17T00:20:10.732 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/3467557048 2023-11-17T23:58:33.059992+0000 2023-11-17T00:20:10.732 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6:0/2971625275 2023-11-17T23:59:01.653365+0000 2023-11-17T00:20:10.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.726+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb2c0679e0 con 0x7fcb2c1094b0 2023-11-17T00:20:10.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.727+0000 7fcb1f7fe700 1 -- 172.21.15.6:0/2391606422 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) v1 ==== 81+0+926 (secure 0 0 0) 0x7fcb20099b50 con 0x7fcb2c1094b0 2023-11-17T00:20:10.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 msgr2=0x7fcb180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:10.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 0x7fcb180a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c109b90 tx=0x7fcb24009470 comp rx=0 tx=0).stop 2023-11-17T00:20:10.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 msgr2=0x7fcb2c1a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:10.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000ef10 tx=0x7fcb2000c3a0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 shutdown_connections 2023-11-17T00:20:10.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fcb180a1340 0x7fcb180a37f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb2c10ea60 0x7fcb2c1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb2c108b00 0x7fcb2c1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 --2- 172.21.15.6:0/2391606422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb2c1094b0 0x7fcb2c1a6df0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:10.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.734+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 >> 172.21.15.6:0/2391606422 conn(0x7fcb2c072910 msgr2=0x7fcb2c06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:10.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.735+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 shutdown_connections 2023-11-17T00:20:10.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:10.735+0000 7fcb345ff700 1 -- 172.21.15.6:0/2391606422 wait complete. 2023-11-17T00:20:10.740 INFO:teuthology.orchestra.run.smithi006.stderr:listed 16 entries 2023-11-17T00:20:10.769 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-17T00:20:10.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:20:10.887 DEBUG:teuthology.orchestra.run.smithi006:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-17T00:20:11.041 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:10 smithi006 ceph-mon[97840]: pgmap v350: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:11.041 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:10 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2391606422' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:20:11.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:20:11.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:20:11.094 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-17T00:20:11.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:20:11.193 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-17T00:20:11.342 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-17T00:20:11.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:20:11.343 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-17T00:20:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:10 smithi087 ceph-mon[131182]: pgmap v350: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:11.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:10 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2391606422' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:20:11.365 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:10 smithi186 ceph-mon[131487]: pgmap v350: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-11-17T00:20:11.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:10 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2391606422' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-17T00:20:11.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-17T00:20:11.486 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-17T00:20:11.671 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-17T00:20:11.684 INFO:tasks.cephadm:Teardown begin 2023-11-17T00:20:11.684 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-17T00:20:11.685 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:20:12.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 -- 172.21.15.6:0/522649019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40000ff180 msgr2=0x7f40000ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:12.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 --2- 172.21.15.6:0/522649019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40000ff180 0x7f40000ff550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009990 tx=0x7f3ff0009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 -- 172.21.15.6:0/522649019 shutdown_connections 2023-11-17T00:20:12.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 --2- 172.21.15.6:0/522649019 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4000104680 0x7f4000104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 --2- 172.21.15.6:0/522649019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40000ff180 0x7f40000ff550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 --2- 172.21.15.6:0/522649019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40000ffa90 0x7f4000104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.881+0000 7f40083ff700 1 -- 172.21.15.6:0/522649019 >> 172.21.15.6:0/522649019 conn(0x7f4000077060 msgr2=0x7f4000077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:12.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.887+0000 7f40083ff700 1 -- 172.21.15.6:0/522649019 shutdown_connections 2023-11-17T00:20:12.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.887+0000 7f40083ff700 1 -- 172.21.15.6:0/522649019 wait complete. 2023-11-17T00:20:12.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.888+0000 7f40083ff700 1 Processor -- start 2023-11-17T00:20:12.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.888+0000 7f40083ff700 1 -- start start 2023-11-17T00:20:12.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:12.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40000ffa90 0x7f40001a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:12.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4000104680 0x7f400019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:12.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400019d010 con 0x7f40000ff180 2023-11-17T00:20:12.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400019d180 con 0x7f40000ffa90 2023-11-17T00:20:12.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.889+0000 7f40083ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400019d340 con 0x7f4000104680 2023-11-17T00:20:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.890+0000 7f400619b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.890+0000 7f400619b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34496/0 (socket says 172.21.15.6:34496) 2023-11-17T00:20:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.890+0000 7f400619b700 1 -- 172.21.15.6:0/2986094558 learned_addr learned my addr 172.21.15.6:0/2986094558 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:12.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 -- 172.21.15.6:0/2986094558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4000104680 msgr2=0x7f400019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:12.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4000104680 0x7f400019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 -- 172.21.15.6:0/2986094558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40000ffa90 msgr2=0x7f40001a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:12.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40000ffa90 0x7f40001a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 -- 172.21.15.6:0/2986094558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00095f0 con 0x7f40000ff180 2023-11-17T00:20:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.891+0000 7f400619b700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000c00 tx=0x7f3ff000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:12.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.892+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff001d070 con 0x7f40000ff180 2023-11-17T00:20:12.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.892+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff000fc90 con 0x7f40000ff180 2023-11-17T00:20:12.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.892+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0017650 con 0x7f40000ff180 2023-11-17T00:20:12.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.893+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4000116a20 con 0x7f40000ff180 2023-11-17T00:20:12.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.893+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4000116890 con 0x7f40000ff180 2023-11-17T00:20:12.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.895+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4000116bb0 con 0x7f40000ff180 2023-11-17T00:20:12.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.898+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3ff0017870 con 0x7f40000ff180 2023-11-17T00:20:12.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.900+0000 7f3ff77fe700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 0x7f3fec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:12.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.901+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9573+0+0 (secure 0 0 0) 0x7f3ff000fe00 con 0x7f40000ff180 2023-11-17T00:20:12.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.901+0000 7f400599a700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 0x7f3fec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:12.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.904+0000 7f400599a700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 0x7f3fec0a37f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40000695f0 tx=0x7f3ffc006d20 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:12.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:12.918+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ff0097bf0 con 0x7f40000ff180 2023-11-17T00:20:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:12 smithi006 ceph-mon[97840]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:20:13.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:12 smithi006 ceph-mon[97840]: osdmap e103: 12 total, 12 up, 12 in 2023-11-17T00:20:13.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:12 smithi087 ceph-mon[131182]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:20:13.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:12 smithi087 ceph-mon[131182]: osdmap e103: 12 total, 12 up, 12 in 2023-11-17T00:20:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:12 smithi186 ceph-mon[131487]: pgmap v351: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-11-17T00:20:13.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:12 smithi186 ceph-mon[131487]: osdmap e103: 12 total, 12 up, 12 in 2023-11-17T00:20:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.416+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4000067af0 con 0x7f40000ff180 2023-11-17T00:20:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.418+0000 7f3ff77fe700 1 -- 172.21.15.6:0/2986094558 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7f3ff0097d90 con 0x7f40000ff180 2023-11-17T00:20:13.420 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:20:13.420 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:20:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 msgr2=0x7f3fec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 0x7f3fec0a37f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40000695f0 tx=0x7f3ffc006d20 comp rx=0 tx=0).stop 2023-11-17T00:20:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 msgr2=0x7f40001a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000c00 tx=0x7f3ff000f7b0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 shutdown_connections 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f3fec0a1340 0x7f3fec0a37f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40000ff180 0x7f40001a2930 secure :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000c00 tx=0x7f3ff000f7b0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4000104680 0x7f400019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 --2- 172.21.15.6:0/2986094558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40000ffa90 0x7f40001a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 >> 172.21.15.6:0/2986094558 conn(0x7f4000077060 msgr2=0x7f40001031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 shutdown_connections 2023-11-17T00:20:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.422+0000 7f40083ff700 1 -- 172.21.15.6:0/2986094558 wait complete. 2023-11-17T00:20:13.429 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:20:13.460 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-17T00:20:13.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.822+0000 7f73cf59e700 1 -- 172.21.15.6:0/1198824723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 msgr2=0x7f73d0107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.822+0000 7f73cf59e700 1 --2- 172.21.15.6:0/1198824723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0107f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009990 tx=0x7f73b8009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.823+0000 7f73cf59e700 1 -- 172.21.15.6:0/1198824723 shutdown_connections 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.823+0000 7f73cf59e700 1 --2- 172.21.15.6:0/1198824723 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 0x7f73d0113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.823+0000 7f73cf59e700 1 --2- 172.21.15.6:0/1198824723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0107f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.823+0000 7f73cf59e700 1 --2- 172.21.15.6:0/1198824723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d0110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.823+0000 7f73cf59e700 1 -- 172.21.15.6:0/1198824723 >> 172.21.15.6:0/1198824723 conn(0x7f73d006c7e0 msgr2=0x7f73d006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:13.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.826+0000 7f73cf59e700 1 -- 172.21.15.6:0/1198824723 shutdown_connections 2023-11-17T00:20:13.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.826+0000 7f73cf59e700 1 -- 172.21.15.6:0/1198824723 wait complete. 2023-11-17T00:20:13.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.829+0000 7f73cf59e700 1 Processor -- start 2023-11-17T00:20:13.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.829+0000 7f73cf59e700 1 -- start start 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 0x7f73d010b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d010b800 con 0x7f73d0111130 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01c2490 con 0x7f73d0107b80 2023-11-17T00:20:13.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.831+0000 7f73cf59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01c25d0 con 0x7f73d0108520 2023-11-17T00:20:13.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73ce59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:13.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:13.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37398/0 (socket says 172.21.15.6:37398) 2023-11-17T00:20:13.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 -- 172.21.15.6:0/2694914327 learned_addr learned my addr 172.21.15.6:0/2694914327 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:13.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73ced9d700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 0x7f73d010b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:13.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 -- 172.21.15.6:0/2694914327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 msgr2=0x7f73d0110a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:13.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0110a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 -- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 msgr2=0x7f73d010b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:13.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 0x7f73d010b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.832+0000 7f73cdd9b700 1 -- 172.21.15.6:0/2694914327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80095f0 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73cdd9b700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f73c000ada0 tx=0x7f73c000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c000f070 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c000ae90 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c00053e0 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d0074180 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.833+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d0073ff0 con 0x7f73d0108520 2023-11-17T00:20:13.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.835+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d0074310 con 0x7f73d0108520 2023-11-17T00:20:13.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.845+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f73c000c980 con 0x7f73d0108520 2023-11-17T00:20:13.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.847+0000 7f73c77fe700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 0x7f73bc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:13.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.848+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9573+0+0 (secure 0 0 0) 0x7f73c00cc6f0 con 0x7f73d0108520 2023-11-17T00:20:13.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.848+0000 7f73ce59c700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 0x7f73bc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:13.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.849+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73c00ccbd0 con 0x7f73d0108520 2023-11-17T00:20:13.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:13.850+0000 7f73ce59c700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 0x7f73bc0a38c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b5c0 tx=0x7f73b8005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:14.223 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:13 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2986094558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.325+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73d0080010 con 0x7f73d0108520 2023-11-17T00:20:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.328+0000 7f73c77fe700 1 -- 172.21.15.6:0/2694914327 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10739 (secure 0 0 0) 0x7f73c0021090 con 0x7f73d0108520 2023-11-17T00:20:14.335 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:20:14.335 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3779928633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3779928633},{"type":"v1","addr":"172.21.15.6:6835","nonce":3779928633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3190561786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":3190561786},{"type":"v1","addr":"172.21.15.6:6837","nonce":3190561786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14508,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6839/3324637103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6838","nonce":3324637103},{"type":"v1","addr":"172.21.15.6:6839","nonce":3324637103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6841/1535787123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6840","nonce":1535787123},{"type":"v1","addr":"172.21.15.6:6841","nonce":1535787123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3460267668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3460267668},{"type":"v1","addr":"172.21.15.186:6833","nonce":3460267668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/2959279613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2959279613},{"type":"v1","addr":"172.21.15.87:6833","nonce":2959279613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1004385520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1004385520},{"type":"v1","addr":"172.21.15.186:6835","nonce":1004385520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2088255632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2088255632},{"type":"v1","addr":"172.21.15.87:6835","nonce":2088255632}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3856911296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":3856911296},{"type":"v1","addr":"172.21.15.186:6837","nonce":3856911296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":26147,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1614644612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1614644612},{"type":"v1","addr":"172.21.15.87:6837","nonce":1614644612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-17T00:06:12.598183+0000","modified":"2023-11-17T00:11:38.445925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24532":{"gid":24532,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6839/143156596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":143156596},{"type":"v1","addr":"172.21.15.186:6839","nonce":143156596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.87:6839/804288298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":804288298},{"type":"v1","addr":"172.21.15.87:6839","nonce":804288298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-17T00:20:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.332+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 msgr2=0x7f73bc0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.332+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 0x7f73bc0a38c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b5c0 tx=0x7f73b8005fb0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.332+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 msgr2=0x7f73d010aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.332+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f73c000ada0 tx=0x7f73c000c330 comp rx=0 tx=0).stop 2023-11-17T00:20:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 shutdown_connections 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f73bc0a1410 0x7f73bc0a38c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73d0111130 0x7f73d010b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73d0107b80 0x7f73d0110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 --2- 172.21.15.6:0/2694914327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f73d0108520 0x7f73d010aa00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 >> 172.21.15.6:0/2694914327 conn(0x7f73d006c7e0 msgr2=0x7f73d007d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 shutdown_connections 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.333+0000 7f73cf59e700 1 -- 172.21.15.6:0/2694914327 wait complete. 2023-11-17T00:20:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 22 2023-11-17T00:20:14.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:13 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2986094558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:14.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:13 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2986094558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:14.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-17T00:20:14.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 -- 172.21.15.6:0/2095057098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 msgr2=0x7f53a410a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:14.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 --2- 172.21.15.6:0/2095057098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a410a370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5398009990 tx=0x7f5398009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 -- 172.21.15.6:0/2095057098 shutdown_connections 2023-11-17T00:20:14.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 --2- 172.21.15.6:0/2095057098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a410a8b0 0x7f53a410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 --2- 172.21.15.6:0/2095057098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53a4101390 0x7f53a4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 --2- 172.21.15.6:0/2095057098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a410a370 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.727+0000 7f53a9fff700 1 -- 172.21.15.6:0/2095057098 >> 172.21.15.6:0/2095057098 conn(0x7f53a4077060 msgr2=0x7f53a4077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:14.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.729+0000 7f53a9fff700 1 -- 172.21.15.6:0/2095057098 shutdown_connections 2023-11-17T00:20:14.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.729+0000 7f53a9fff700 1 -- 172.21.15.6:0/2095057098 wait complete. 2023-11-17T00:20:14.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.730+0000 7f53a9fff700 1 Processor -- start 2023-11-17T00:20:14.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 -- start start 2023-11-17T00:20:14.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a4101390 0x7f53a4194280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:14.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:14.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53a410a8b0 0x7f53a4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:14.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4199260 con 0x7f53a4101390 2023-11-17T00:20:14.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41993d0 con 0x7f53a4101ca0 2023-11-17T00:20:14.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.731+0000 7f53a9fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4199590 con 0x7f53a410a8b0 2023-11-17T00:20:14.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:14.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40316/0 (socket says 172.21.15.6:40316) 2023-11-17T00:20:14.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 -- 172.21.15.6:0/269751557 learned_addr learned my addr 172.21.15.6:0/269751557 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:14.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 -- 172.21.15.6:0/269751557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53a410a8b0 msgr2=0x7f53a4198c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53a410a8b0 0x7f53a4198c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 -- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a4101390 msgr2=0x7f53a4194280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a4101390 0x7f53a4194280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.732+0000 7f53a2ffd700 1 -- 172.21.15.6:0/269751557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980095f0 con 0x7f53a4101ca0 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.733+0000 7f53a2ffd700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5398005910 tx=0x7f53980057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.733+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539801d070 con 0x7f53a4101ca0 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.733+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a410e200 con 0x7f53a4101ca0 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.733+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539800fb80 con 0x7f53a4101ca0 2023-11-17T00:20:14.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.734+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a410e070 con 0x7f53a4101ca0 2023-11-17T00:20:14.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.734+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539801d070 con 0x7f53a4101ca0 2023-11-17T00:20:14.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.735+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a410e390 con 0x7f53a4101ca0 2023-11-17T00:20:14.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.753+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f539800fcf0 con 0x7f53a4101ca0 2023-11-17T00:20:14.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.755+0000 7f53a0ff9700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 0x7f538c0a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:14.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.755+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9573+0+0 (secure 0 0 0) 0x7f53980cdf70 con 0x7f53a4101ca0 2023-11-17T00:20:14.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.756+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f53980ce3f0 con 0x7f53a4101ca0 2023-11-17T00:20:14.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.771+0000 7f53a37fe700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 0x7f538c0a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:14.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:14.772+0000 7f53a37fe700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 0x7f538c0a7cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5394009af0 tx=0x7f5394009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:15.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.104+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a41955d0 con 0x7f53a4101ca0 2023-11-17T00:20:15.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.107+0000 7f53a0ff9700 1 -- 172.21.15.6:0/269751557 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25216 (secure 0 0 0) 0x7f5398097020 con 0x7f53a4101ca0 2023-11-17T00:20:15.110 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-17T00:20:15.110 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":103,"fsid":"dbc2f8cc-84db-11ee-8dbe-212e2dc638e7","created":"2023-11-16T23:58:00.220236+0000","modified":"2023-11-17T00:20:11.936794+0000","last_up_change":"2023-11-17T00:05:20.306797+0000","last_in_change":"2023-11-17T00:05:02.751065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-17T00:01:28.259100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-17T00:06:11.024961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-17T00:06:12.046799+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-17T00:06:15.576749+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf2da7d9-d6af-4949-9a5f-d392c2dcc273","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6803","nonce":3863583460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6805","nonce":3863583460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6809","nonce":3863583460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3863583460},{"type":"v1","addr":"172.21.15.6:6807","nonce":3863583460}]},"public_addr":"172.21.15.6:6803/3863583460","cluster_addr":"172.21.15.6:6805/3863583460","heartbeat_back_addr":"172.21.15.6:6809/3863583460","heartbeat_front_addr":"172.21.15.6:6807/3863583460","state":["exists","up"]},{"osd":1,"uuid":"cbdc1ecd-4466-47fc-8510-60d676144e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6801","nonce":4270277324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6803","nonce":4270277324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6807","nonce":4270277324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":4270277324},{"type":"v1","addr":"172.21.15.87:6805","nonce":4270277324}]},"public_addr":"172.21.15.87:6801/4270277324","cluster_addr":"172.21.15.87:6803/4270277324","heartbeat_back_addr":"172.21.15.87:6807/4270277324","heartbeat_front_addr":"172.21.15.87:6805/4270277324","state":["exists","up"]},{"osd":2,"uuid":"17362d0b-f379-4a0e-a815-aec6d56732e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6801","nonce":1950482303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6803","nonce":1950482303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6807","nonce":1950482303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1950482303},{"type":"v1","addr":"172.21.15.186:6805","nonce":1950482303}]},"public_addr":"172.21.15.186:6801/1950482303","cluster_addr":"172.21.15.186:6803/1950482303","heartbeat_back_addr":"172.21.15.186:6807/1950482303","heartbeat_front_addr":"172.21.15.186:6805/1950482303","state":["exists","up"]},{"osd":3,"uuid":"d94de1b8-5e3a-4344-b794-04389000f0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6811","nonce":1124657350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6813","nonce":1124657350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6817","nonce":1124657350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1124657350},{"type":"v1","addr":"172.21.15.6:6815","nonce":1124657350}]},"public_addr":"172.21.15.6:6811/1124657350","cluster_addr":"172.21.15.6:6813/1124657350","heartbeat_back_addr":"172.21.15.6:6817/1124657350","heartbeat_front_addr":"172.21.15.6:6815/1124657350","state":["exists","up"]},{"osd":4,"uuid":"17ec1e9b-999c-47e6-87ca-3405277892f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6809","nonce":31913471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6811","nonce":31913471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6815","nonce":31913471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":31913471},{"type":"v1","addr":"172.21.15.87:6813","nonce":31913471}]},"public_addr":"172.21.15.87:6809/31913471","cluster_addr":"172.21.15.87:6811/31913471","heartbeat_back_addr":"172.21.15.87:6815/31913471","heartbeat_front_addr":"172.21.15.87:6813/31913471","state":["exists","up"]},{"osd":5,"uuid":"f8e5ab3b-1735-4f77-82c8-a374f7fab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6809","nonce":2177950803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6811","nonce":2177950803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6815","nonce":2177950803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2177950803},{"type":"v1","addr":"172.21.15.186:6813","nonce":2177950803}]},"public_addr":"172.21.15.186:6809/2177950803","cluster_addr":"172.21.15.186:6811/2177950803","heartbeat_back_addr":"172.21.15.186:6815/2177950803","heartbeat_front_addr":"172.21.15.186:6813/2177950803","state":["exists","up"]},{"osd":6,"uuid":"8efcc3e3-0f96-4248-8eac-6f5041c0b4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6819","nonce":1113025990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6821","nonce":1113025990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6825","nonce":1113025990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":1113025990},{"type":"v1","addr":"172.21.15.6:6823","nonce":1113025990}]},"public_addr":"172.21.15.6:6819/1113025990","cluster_addr":"172.21.15.6:6821/1113025990","heartbeat_back_addr":"172.21.15.6:6825/1113025990","heartbeat_front_addr":"172.21.15.6:6823/1113025990","state":["exists","up"]},{"osd":7,"uuid":"9282e04d-a458-456a-ac74-a942aa06cdc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6817","nonce":1542327799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6819","nonce":1542327799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6823","nonce":1542327799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1542327799},{"type":"v1","addr":"172.21.15.87:6821","nonce":1542327799}]},"public_addr":"172.21.15.87:6817/1542327799","cluster_addr":"172.21.15.87:6819/1542327799","heartbeat_back_addr":"172.21.15.87:6823/1542327799","heartbeat_front_addr":"172.21.15.87:6821/1542327799","state":["exists","up"]},{"osd":8,"uuid":"02359124-89e5-4f4a-a547-b5fe9e03d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6817","nonce":4126523122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6819","nonce":4126523122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6823","nonce":4126523122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4126523122},{"type":"v1","addr":"172.21.15.186:6821","nonce":4126523122}]},"public_addr":"172.21.15.186:6817/4126523122","cluster_addr":"172.21.15.186:6819/4126523122","heartbeat_back_addr":"172.21.15.186:6823/4126523122","heartbeat_front_addr":"172.21.15.186:6821/4126523122","state":["exists","up"]},{"osd":9,"uuid":"75b7d811-9e00-427c-a0b7-e300fd19a9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6827","nonce":1254890156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6829","nonce":1254890156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6833","nonce":1254890156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1254890156},{"type":"v1","addr":"172.21.15.6:6831","nonce":1254890156}]},"public_addr":"172.21.15.6:6827/1254890156","cluster_addr":"172.21.15.6:6829/1254890156","heartbeat_back_addr":"172.21.15.6:6833/1254890156","heartbeat_front_addr":"172.21.15.6:6831/1254890156","state":["exists","up"]},{"osd":10,"uuid":"5acfeda0-5332-4020-bcc7-2173606e96ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6825","nonce":1132375941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6827","nonce":1132375941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6831","nonce":1132375941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1132375941},{"type":"v1","addr":"172.21.15.87:6829","nonce":1132375941}]},"public_addr":"172.21.15.87:6825/1132375941","cluster_addr":"172.21.15.87:6827/1132375941","heartbeat_back_addr":"172.21.15.87:6831/1132375941","heartbeat_front_addr":"172.21.15.87:6829/1132375941","state":["exists","up"]},{"osd":11,"uuid":"f058968e-7426-4a1c-917e-09beb2be51ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6825","nonce":3407026982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6827","nonce":3407026982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6831","nonce":3407026982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":3407026982},{"type":"v1","addr":"172.21.15.186:6829","nonce":3407026982}]},"public_addr":"172.21.15.186:6825/3407026982","cluster_addr":"172.21.15.186:6827/3407026982","heartbeat_back_addr":"172.21.15.186:6831/3407026982","heartbeat_front_addr":"172.21.15.186:6829/3407026982","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:00:42.099739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:04.018837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:25.401499+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:01:57.943837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:20.303311+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:02:42.230172+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:10.712218+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:33.110356+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:03:55.153577+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:35.228980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:04:57.201721+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-17T00:05:18.959826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:6801/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6800/879660796":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1767647821":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/2273227531":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/1821215069":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/4157530954":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:6800/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/4272143857":"2023-11-18T00:08:53.335648+0000","172.21.15.6:0/3407461840":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3069406217":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/3662483346":"2023-11-18T00:08:53.335648+0000","172.21.15.6:6801/3715783566":"2023-11-17T23:58:33.059992+0000","172.21.15.6:6801/610057204":"2023-11-17T23:59:01.653365+0000","172.21.15.6:0/3467557048":"2023-11-17T23:58:33.059992+0000","172.21.15.6:0/2971625275":"2023-11-17T23:59:01.653365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":2},{"begin":17,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":2},{"begin":17,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-17T00:20:15.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.113+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 msgr2=0x7f538c0a7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:15.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.113+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 0x7f538c0a7cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5394009af0 tx=0x7f5394009260 comp rx=0 tx=0).stop 2023-11-17T00:20:15.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.113+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 msgr2=0x7f53a41947c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:15.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.113+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5398005910 tx=0x7f53980057f0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 shutdown_connections 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f538c0a5810 0x7f538c0a7cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a4101390 0x7f53a4194280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53a410a8b0 0x7f53a4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 --2- 172.21.15.6:0/269751557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53a4101ca0 0x7f53a41947c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 >> 172.21.15.6:0/269751557 conn(0x7f53a4077060 msgr2=0x7f53a40fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:15.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 shutdown_connections 2023-11-17T00:20:15.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.114+0000 7f53a9fff700 1 -- 172.21.15.6:0/269751557 wait complete. 2023-11-17T00:20:15.155 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-17T00:20:15.267 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:14 smithi006 ceph-mon[97840]: pgmap v353: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2023-11-17T00:20:15.267 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:14 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2694914327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:15.354 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:14 smithi087 ceph-mon[131182]: pgmap v353: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2023-11-17T00:20:15.355 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:14 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2694914327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:14 smithi186 ceph-mon[131487]: pgmap v353: 129 pgs: 21 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 43 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2023-11-17T00:20:15.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:14 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2694914327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-17T00:20:15.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.519+0000 7facdcf9d700 1 -- 172.21.15.6:0/2573001189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8102530 msgr2=0x7facd8102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:15.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.519+0000 7facdcf9d700 1 --2- 172.21.15.6:0/2573001189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8102530 0x7facd8102900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7facc8009f90 tx=0x7facc800b330 comp rx=0 tx=0).stop 2023-11-17T00:20:15.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 -- 172.21.15.6:0/2573001189 shutdown_connections 2023-11-17T00:20:15.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 --2- 172.21.15.6:0/2573001189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8107570 0x7facd810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 --2- 172.21.15.6:0/2573001189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8102530 0x7facd8102900 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 --2- 172.21.15.6:0/2573001189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102ed0 0x7facd8106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 -- 172.21.15.6:0/2573001189 >> 172.21.15.6:0/2573001189 conn(0x7facd80fddc0 msgr2=0x7facd81001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:15.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 -- 172.21.15.6:0/2573001189 shutdown_connections 2023-11-17T00:20:15.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.520+0000 7facdcf9d700 1 -- 172.21.15.6:0/2573001189 wait complete. 2023-11-17T00:20:15.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.522+0000 7facdcf9d700 1 Processor -- start 2023-11-17T00:20:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.522+0000 7facdcf9d700 1 -- start start 2023-11-17T00:20:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 0x7facd8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:15.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8107570 0x7facd819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819d510 con 0x7facd8102530 2023-11-17T00:20:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819d680 con 0x7facd8107570 2023-11-17T00:20:15.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.523+0000 7facdcf9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819d840 con 0x7facd8102ed0 2023-11-17T00:20:15.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facd77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 0x7facd8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:15.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:15.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facd7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8107570 0x7facd819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:15.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37422/0 (socket says 172.21.15.6:37422) 2023-11-17T00:20:15.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 -- 172.21.15.6:0/4000795039 learned_addr learned my addr 172.21.15.6:0/4000795039 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:15.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 -- 172.21.15.6:0/4000795039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8107570 msgr2=0x7facd819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:15.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8107570 0x7facd819ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.524+0000 7facceffd700 1 -- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 msgr2=0x7facd8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.525+0000 7facceffd700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 0x7facd8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.525+0000 7facceffd700 1 -- 172.21.15.6:0/4000795039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009c40 con 0x7facd8102ed0 2023-11-17T00:20:15.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.525+0000 7facd77fe700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 0x7facd8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:20:15.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.526+0000 7facceffd700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facc000c8c0 tx=0x7facc000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.530+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0004220 con 0x7facd8102ed0 2023-11-17T00:20:15.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.530+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc0007dc0 con 0x7facd8102ed0 2023-11-17T00:20:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.530+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0004e40 con 0x7facd8102ed0 2023-11-17T00:20:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.530+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd81123d0 con 0x7facd8102ed0 2023-11-17T00:20:15.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.531+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd8112240 con 0x7facd8102ed0 2023-11-17T00:20:15.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.533+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd8112560 con 0x7facd8102ed0 2023-11-17T00:20:15.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.543+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7facc0007900 con 0x7facd8102ed0 2023-11-17T00:20:15.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.545+0000 7facd57fa700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 0x7facc40a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:15.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.545+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9573+0+0 (secure 0 0 0) 0x7facc00ccd70 con 0x7facd8102ed0 2023-11-17T00:20:15.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.546+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7facc00cd1f0 con 0x7facd8102ed0 2023-11-17T00:20:15.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.546+0000 7facd77fe700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 0x7facc40a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:15.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:15.550+0000 7facd77fe700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 0x7facc40a38b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facc80060b0 tx=0x7facc8006020 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:16.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:16.062+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7facd8067970 con 0x7facd8102ed0 2023-11-17T00:20:16.224 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:15 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/269751557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:16.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:15 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/269751557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:16.352 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-17T00:20:16.352 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-17T00:20:16.353 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-17T00:20:16.353 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: -4> 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-17T00:20:16.353 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: -3> 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-17T00:20:16.353 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: -1> 2023-11-17T00:20:16.116+0000 7f5b02893700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-17T00:20:16.354 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: ignoring --setuser ceph since I am not root 2023-11-17T00:20:16.354 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: ignoring --setgroup ceph since I am not root 2023-11-17T00:20:16.354 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-mds[156492]: -- 172.21.15.87:0/1388042551 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591e912b520 con 0x5591e9100800 2023-11-17T00:20:16.354 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: starting mds.b at 2023-11-17T00:20:16.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:15 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/269751557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-17T00:20:17.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.021+0000 7facd57fa700 1 -- 172.21.15.6:0/4000795039 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7facc001d070 con 0x7facd8102ed0 2023-11-17T00:20:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 msgr2=0x7facc40a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 0x7facc40a38b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facc80060b0 tx=0x7facc8006020 comp rx=0 tx=0).stop 2023-11-17T00:20:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 msgr2=0x7facd8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facc000c8c0 tx=0x7facc000cc80 comp rx=0 tx=0).stop 2023-11-17T00:20:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 shutdown_connections 2023-11-17T00:20:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7facc40a1400 0x7facc40a38b0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facc80060b0 tx=0x7facc8006020 comp rx=0 tx=0).stop 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7facd8102530 0x7facd8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd8107570 0x7facd819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 --2- 172.21.15.6:0/4000795039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102ed0 0x7facd8198a70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 >> 172.21.15.6:0/4000795039 conn(0x7facd80fddc0 msgr2=0x7facd8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 shutdown_connections 2023-11-17T00:20:17.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.026+0000 7facdcf9d700 1 -- 172.21.15.6:0/4000795039 wait complete. 2023-11-17T00:20:17.033 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-17T00:20:17.062 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-17T00:20:17.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-mon[131182]: pgmap v354: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 14 KiB/s wr, 4 op/s 2023-11-17T00:20:17.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/4000795039' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:16 smithi087 ceph-mon[131182]: osdmap e104: 12 total, 12 up, 12 in 2023-11-17T00:20:17.365 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: ignoring --setuser ceph since I am not root 2023-11-17T00:20:17.366 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: ignoring --setgroup ceph since I am not root 2023-11-17T00:20:17.366 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: starting mds.i at 2023-11-17T00:20:17.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:16 smithi186 ceph-mon[131487]: pgmap v354: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 14 KiB/s wr, 4 op/s 2023-11-17T00:20:17.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:16 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/4000795039' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:16 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:16 smithi186 ceph-mon[131487]: osdmap e104: 12 total, 12 up, 12 in 2023-11-17T00:20:17.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 -- 172.21.15.6:0/1393007500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 msgr2=0x7fd5300b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 --2- 172.21.15.6:0/1393007500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd5300b2e30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5380640e0 tx=0x7fd53806f7b0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 -- 172.21.15.6:0/1393007500 shutdown_connections 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 --2- 172.21.15.6:0/1393007500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd5300b2e30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 --2- 172.21.15.6:0/1393007500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a62b0 0x7fd5300a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 --2- 172.21.15.6:0/1393007500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300a52c0 0x7fd5300a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 -- 172.21.15.6:0/1393007500 >> 172.21.15.6:0/1393007500 conn(0x7fd53001a2e0 msgr2=0x7fd53001a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 -- 172.21.15.6:0/1393007500 shutdown_connections 2023-11-17T00:20:17.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.435+0000 7fd53f39d700 1 -- 172.21.15.6:0/1393007500 wait complete. 2023-11-17T00:20:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.437+0000 7fd53f39d700 1 Processor -- start 2023-11-17T00:20:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.437+0000 7fd53f39d700 1 -- start start 2023-11-17T00:20:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd530147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300af070 0x7fd5300af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300afb40 con 0x7fd5300af070 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300afcb0 con 0x7fd5300b09a0 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53f39d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300afe70 con 0x7fd5300a52c0 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:37440/0 (socket says 172.21.15.6:37440) 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53e39b700 1 -- 172.21.15.6:0/219027076 learned_addr learned my addr 172.21.15.6:0/219027076 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:17.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.438+0000 7fd53db9a700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd530147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:17.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53eb9c700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300af070 0x7fd5300af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:17.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53e39b700 1 -- 172.21.15.6:0/219027076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 msgr2=0x7fd530147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:17.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53e39b700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd530147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53e39b700 1 -- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300af070 msgr2=0x7fd5300af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:17.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53e39b700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300af070 0x7fd5300af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:17.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.439+0000 7fd53e39b700 1 -- 172.21.15.6:0/219027076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534009520 con 0x7fd5300a52c0 2023-11-17T00:20:17.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.444+0000 7fd53e39b700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd53400eb30 tx=0x7fd53400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:17.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.444+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534009880 con 0x7fd5300a52c0 2023-11-17T00:20:17.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.444+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5340044e0 con 0x7fd5300a52c0 2023-11-17T00:20:17.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.444+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534003e90 con 0x7fd5300a52c0 2023-11-17T00:20:17.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.444+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53804f760 con 0x7fd5300a52c0 2023-11-17T00:20:17.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.445+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300b64d0 con 0x7fd5300a52c0 2023-11-17T00:20:17.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.446+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5300b6340 con 0x7fd5300a52c0 2023-11-17T00:20:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.461+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd534004000 con 0x7fd5300a52c0 2023-11-17T00:20:17.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.465+0000 7fd52f7fe700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 0x7fd5240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.466+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9571+0+0 (secure 0 0 0) 0x7fd534014070 con 0x7fd5300a52c0 2023-11-17T00:20:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.466+0000 7fd53db9a700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 0x7fd5240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.466+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd5340cd910 con 0x7fd5300a52c0 2023-11-17T00:20:17.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.467+0000 7fd53db9a700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 0x7fd5240a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd530144280 tx=0x7fd528009190 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:17.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a[97816]: 2023-11-17T00:20:16.988+0000 7fc306680700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-17T00:20:17.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:16 smithi006 ceph-mon[97840]: pgmap v354: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 14 KiB/s wr, 4 op/s 2023-11-17T00:20:17.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:16 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/4000795039' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:16 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-17T00:20:17.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:16 smithi006 ceph-mon[97840]: osdmap e104: 12 total, 12 up, 12 in 2023-11-17T00:20:17.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:17.977+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5300b6660 con 0x7fd5300a52c0 2023-11-17T00:20:18.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.025+0000 7fd52f7fe700 1 -- 172.21.15.6:0/219027076 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd534096670 con 0x7fd5300a52c0 2023-11-17T00:20:18.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.029+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 msgr2=0x7fd5240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:18.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.029+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 0x7fd5240a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd530144280 tx=0x7fd528009190 comp rx=0 tx=0).stop 2023-11-17T00:20:18.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.029+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 msgr2=0x7fd5301477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:18.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.029+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd53400eb30 tx=0x7fd53400ee40 comp rx=0 tx=0).stop 2023-11-17T00:20:18.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 shutdown_connections 2023-11-17T00:20:18.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fd5240a1340 0x7fd5240a37f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5300af070 0x7fd5300af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5300a52c0 0x7fd5301477f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 --2- 172.21.15.6:0/219027076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5300b09a0 0x7fd530147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.030+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 >> 172.21.15.6:0/219027076 conn(0x7fd53001a2e0 msgr2=0x7fd5300a6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:18.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.031+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 shutdown_connections 2023-11-17T00:20:18.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.031+0000 7fd53f39d700 1 -- 172.21.15.6:0/219027076 wait complete. 2023-11-17T00:20:18.068 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-17T00:20:18.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-17T00:20:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-17T00:20:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-17T00:20:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.87:6838/4250165701,v1:172.21.15.87:6839/4250165701] up:boot 2023-11-17T00:20:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: fsmap cephfs:0/1 11 up:standby, 1 failed 2023-11-17T00:20:18.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:20:18.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/219027076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.353 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:17 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.87:6838/4250165701,v1:172.21.15.87:6839/4250165701] up:boot 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: fsmap cephfs:0/1 11 up:standby, 1 failed 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/219027076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:17 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 -- 172.21.15.6:0/590253757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 msgr2=0x7fc8c01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/590253757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01029b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009990 tx=0x7fc8b4009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 -- 172.21.15.6:0/590253757 shutdown_connections 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/590253757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8c0108570 0x7fc8c0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/590253757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0102ef0 0x7fc8c010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/590253757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01029b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.443+0000 7fc8c61ff700 1 -- 172.21.15.6:0/590253757 >> 172.21.15.6:0/590253757 conn(0x7fc8c00fdfe0 msgr2=0x7fc8c01003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:18.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.445+0000 7fc8c61ff700 1 -- 172.21.15.6:0/590253757 shutdown_connections 2023-11-17T00:20:18.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.445+0000 7fc8c61ff700 1 -- 172.21.15.6:0/590253757 wait complete. 2023-11-17T00:20:18.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.446+0000 7fc8c61ff700 1 Processor -- start 2023-11-17T00:20:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.447+0000 7fc8c61ff700 1 -- start start 2023-11-17T00:20:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.447+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.447+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8c0102ef0 0x7fc8c0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.447+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0108570 0x7fc8c019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:18.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8c61ff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019d5c0 con 0x7fc8c0102ef0 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:40366/0 (socket says 172.21.15.6:40366) 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8c61ff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019d730 con 0x7fc8c0102540 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 -- 172.21.15.6:0/1029092482 learned_addr learned my addr 172.21.15.6:0/1029092482 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019d8f0 con 0x7fc8c0108570 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bffff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0108570 0x7fc8c019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:18.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 -- 172.21.15.6:0/1029092482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0108570 msgr2=0x7fc8c019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0108570 0x7fc8c019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 -- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8c0102ef0 msgr2=0x7fc8c0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8c0102ef0 0x7fc8c0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.448+0000 7fc8bf7fe700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40095f0 con 0x7fc8c0102540 2023-11-17T00:20:18.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.449+0000 7fc8bf7fe700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000d950 tx=0x7fc8b000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.449+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0009940 con 0x7fc8c0102540 2023-11-17T00:20:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.449+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c0112590 con 0x7fc8c0102540 2023-11-17T00:20:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.450+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c0112400 con 0x7fc8c0102540 2023-11-17T00:20:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.450+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b0010460 con 0x7fc8c0102540 2023-11-17T00:20:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.450+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000f620 con 0x7fc8c0102540 2023-11-17T00:20:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.451+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac005310 con 0x7fc8c0102540 2023-11-17T00:20:18.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.455+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc8b0009aa0 con 0x7fc8c0102540 2023-11-17T00:20:18.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.457+0000 7fc8bcff9700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 0x7fc8a80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:18.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.457+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9571+0+0 (secure 0 0 0) 0x7fc8b00ccac0 con 0x7fc8c0102540 2023-11-17T00:20:18.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.460+0000 7fc8beffd700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 0x7fc8a80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:18.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.461+0000 7fc8beffd700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 0x7fc8a80a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40098e0 tx=0x7fc8b4005fd0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:18.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.461+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc8b0095ab0 con 0x7fc8c0102540 2023-11-17T00:20:18.478 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-17T00:20:18.478 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-17T00:20:18.478 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-17T00:20:18.478 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.87:6838/4250165701,v1:172.21.15.87:6839/4250165701] up:boot 2023-11-17T00:20:18.479 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: fsmap cephfs:0/1 11 up:standby, 1 failed 2023-11-17T00:20:18.479 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-17T00:20:18.479 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/219027076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.479 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:17 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-17T00:20:18.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:18.870+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8ac005180 con 0x7fc8c0102540 2023-11-17T00:20:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: pgmap v356: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:20:19.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-17T00:20:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-17T00:20:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: Cluster is now healthy 2023-11-17T00:20:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: mds.? [v2:172.21.15.186:6838/4044673394,v1:172.21.15.186:6839/4044673394] up:boot 2023-11-17T00:20:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:20:19.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: fsmap 12 up:standby 2023-11-17T00:20:19.369 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:20:19.369 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/1029092482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.369 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:19 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.370 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: pgmap v356: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:20:19.370 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-17T00:20:19.370 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-17T00:20:19.370 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: Cluster is now healthy 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: mds.? [v2:172.21.15.186:6838/4044673394,v1:172.21.15.186:6839/4044673394] up:boot 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: fsmap 12 up:standby 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/1029092482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:19 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: pgmap v356: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2023-11-17T00:20:19.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: Cluster is now healthy 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: mds.? [v2:172.21.15.186:6838/4044673394,v1:172.21.15.186:6839/4044673394] up:boot 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: fsmap 12 up:standby 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: from='mgr.25469 172.21.15.6:0/3523761454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-17T00:20:19.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/1029092482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.475 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:18 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:19.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.671+0000 7fc8bcff9700 1 -- 172.21.15.6:0/1029092482 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8b00160d0 con 0x7fc8c0102540 2023-11-17T00:20:19.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 msgr2=0x7fc8a80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:19.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 0x7fc8a80a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40098e0 tx=0x7fc8b4005fd0 comp rx=0 tx=0).stop 2023-11-17T00:20:19.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 msgr2=0x7fc8c01985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:19.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000d950 tx=0x7fc8b000dc60 comp rx=0 tx=0).stop 2023-11-17T00:20:19.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 shutdown_connections 2023-11-17T00:20:19.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7fc8a80a1340 0x7fc8a80a37f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:19.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8c0108570 0x7fc8c019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:19.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c0102540 0x7fc8c01985e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:19.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 --2- 172.21.15.6:0/1029092482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8c0102ef0 0x7fc8c0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:19.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.675+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 >> 172.21.15.6:0/1029092482 conn(0x7fc8c00fdfe0 msgr2=0x7fc8c00ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:19.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.676+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 shutdown_connections 2023-11-17T00:20:19.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:19.676+0000 7fc8c61ff700 1 -- 172.21.15.6:0/1029092482 wait complete. 2023-11-17T00:20:19.681 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2023-11-17T00:20:19.714 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-17T00:20:20.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 -- 172.21.15.6:0/3617691548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 msgr2=0x7f6ee4101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:20.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/3617691548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee4101020 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40099c0 tx=0x7f6ed4009cd0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 -- 172.21.15.6:0/3617691548 shutdown_connections 2023-11-17T00:20:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/3617691548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ee4105560 0x7f6ee410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/3617691548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee4101020 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/3617691548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 0x7f6ee4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 -- 172.21.15.6:0/3617691548 >> 172.21.15.6:0/3617691548 conn(0x7f6ee4077500 msgr2=0x7f6ee4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.101+0000 7f6eea7ff700 1 -- 172.21.15.6:0/3617691548 shutdown_connections 2023-11-17T00:20:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.102+0000 7f6eea7ff700 1 -- 172.21.15.6:0/3617691548 wait complete. 2023-11-17T00:20:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.103+0000 7f6eea7ff700 1 Processor -- start 2023-11-17T00:20:20.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.103+0000 7f6eea7ff700 1 -- start start 2023-11-17T00:20:20.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.104+0000 7f6eea7ff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:20.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.104+0000 7f6eea7ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 0x7f6ee4196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:20.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.104+0000 7f6eea7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ee4105560 0x7f6ee419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:20.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.104+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:20.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.6:54768/0 (socket says 172.21.15.6:54768) 2023-11-17T00:20:20.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 -- 172.21.15.6:0/467244918 learned_addr learned my addr 172.21.15.6:0/467244918 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:20.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee37fe700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 0x7f6ee4196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:20.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419b3c0 con 0x7f6ee41015f0 2023-11-17T00:20:20.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419b530 con 0x7f6ee4100c50 2023-11-17T00:20:20.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419b6f0 con 0x7f6ee4105560 2023-11-17T00:20:20.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 -- 172.21.15.6:0/467244918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ee4105560 msgr2=0x7f6ee419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:20.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ee4105560 0x7f6ee419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 -- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 msgr2=0x7f6ee4196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:20.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.105+0000 7f6ee3fff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 0x7f6ee4196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:20.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.106+0000 7f6ee3fff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc009520 con 0x7f6ee4100c50 2023-11-17T00:20:20.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.107+0000 7f6ee3fff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009990 tx=0x7f6ed401e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:20.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.108+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed401ee00 con 0x7f6ee4100c50 2023-11-17T00:20:20.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.108+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed40095f0 con 0x7f6ee4100c50 2023-11-17T00:20:20.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.108+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41103e0 con 0x7f6ee4100c50 2023-11-17T00:20:20.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.108+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed40054f0 con 0x7f6ee4100c50 2023-11-17T00:20:20.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.108+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400f460 con 0x7f6ee4100c50 2023-11-17T00:20:20.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.111+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4005310 con 0x7f6ee4100c50 2023-11-17T00:20:20.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.113+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6ed4004f00 con 0x7f6ee4100c50 2023-11-17T00:20:20.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.115+0000 7f6ee17fa700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 0x7f6ed00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:20.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.116+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9017+0+0 (secure 0 0 0) 0x7f6ed40d26b0 con 0x7f6ee4100c50 2023-11-17T00:20:20.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.116+0000 7f6ee37fe700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 0x7f6ed00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:20.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.118+0000 7f6ee37fe700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 0x7f6ed00a3870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ee41979f0 tx=0x7f6ecc009260 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:20.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.121+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ed409b900 con 0x7f6ee4100c50 2023-11-17T00:20:20.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:20.551+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ec4005180 con 0x7f6ee4100c50 2023-11-17T00:20:20.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:20 smithi006 ceph-mon[97840]: pgmap v357: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:20:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:20 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:20 smithi006 ceph-mon[97840]: osdmap e105: 12 total, 12 up, 12 in 2023-11-17T00:20:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:20 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/467244918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:20.974 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:20 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:21.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.014+0000 7f6ee17fa700 1 -- 172.21.15.6:0/467244918 <== mon.2 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6ed409ef20 con 0x7f6ee4100c50 2023-11-17T00:20:21.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 msgr2=0x7f6ed00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:21.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 0x7f6ed00a3870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ee41979f0 tx=0x7f6ecc009260 comp rx=0 tx=0).stop 2023-11-17T00:20:21.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 msgr2=0x7f6ee41963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:21.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009990 tx=0x7f6ed401e710 comp rx=0 tx=0).stop 2023-11-17T00:20:21.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 shutdown_connections 2023-11-17T00:20:21.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f6ed00a13c0 0x7f6ed00a3870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ee4105560 0x7f6ee419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee4100c50 0x7f6ee41963e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 --2- 172.21.15.6:0/467244918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ee41015f0 0x7f6ee4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 >> 172.21.15.6:0/467244918 conn(0x7f6ee4077500 msgr2=0x7f6ee40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.019+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 shutdown_connections 2023-11-17T00:20:21.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.020+0000 7f6eea7ff700 1 -- 172.21.15.6:0/467244918 wait complete. 2023-11-17T00:20:21.030 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2023-11-17T00:20:21.057 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-17T00:20:21.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:20 smithi087 ceph-mon[131182]: pgmap v357: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:20:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:20 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:20 smithi087 ceph-mon[131182]: osdmap e105: 12 total, 12 up, 12 in 2023-11-17T00:20:21.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:20 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/467244918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:21.103 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:20 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:21.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:20 smithi186 ceph-mon[131487]: pgmap v357: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 4.9 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 383 B/s wr, 2 op/s 2023-11-17T00:20:21.118 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:20 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:21.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:20 smithi186 ceph-mon[131487]: osdmap e105: 12 total, 12 up, 12 in 2023-11-17T00:20:21.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:20 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/467244918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:21.119 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:20 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:21.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.444+0000 7f86a359e700 1 -- 172.21.15.6:0/2185977931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c09a460 msgr2=0x7f869c09a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:21.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.444+0000 7f86a359e700 1 --2- 172.21.15.6:0/2185977931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c09a460 0x7f869c09a830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8694009990 tx=0x7f8694009ca0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.448+0000 7f86a359e700 1 -- 172.21.15.6:0/2185977931 shutdown_connections 2023-11-17T00:20:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.448+0000 7f86a359e700 1 --2- 172.21.15.6:0/2185977931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c0968c0 0x7f869c0a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.448+0000 7f86a359e700 1 --2- 172.21.15.6:0/2185977931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c09a460 0x7f869c09a830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.448+0000 7f86a359e700 1 --2- 172.21.15.6:0/2185977931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c095f10 0x7f869c096380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.448+0000 7f86a359e700 1 -- 172.21.15.6:0/2185977931 >> 172.21.15.6:0/2185977931 conn(0x7f869c00b4e0 msgr2=0x7f869c00b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.449+0000 7f86a359e700 1 -- 172.21.15.6:0/2185977931 shutdown_connections 2023-11-17T00:20:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.450+0000 7f86a359e700 1 -- 172.21.15.6:0/2185977931 wait complete. 2023-11-17T00:20:21.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.451+0000 7f86a359e700 1 Processor -- start 2023-11-17T00:20:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.453+0000 7f86a359e700 1 -- start start 2023-11-17T00:20:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 0x7f869c12d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:21.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:21.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a1d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.6:33500/0 (socket says 172.21.15.6:33500) 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 0x7f869c130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1315e0 con 0x7f869c095f10 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c131750 con 0x7f869c09a460 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a359e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c131910 con 0x7f869c0968c0 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.454+0000 7f86a1d9b700 1 -- 172.21.15.6:0/2906139099 learned_addr learned my addr 172.21.15.6:0/2906139099 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-17T00:20:21.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.455+0000 7f86a259c700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 0x7f869c12d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:21.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.455+0000 7f86a2d9d700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 0x7f869c130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:21.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 -- 172.21.15.6:0/2906139099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 msgr2=0x7f869c130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:21.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 0x7f869c130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 -- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 msgr2=0x7f869c12d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 0x7f869c12d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 -- 172.21.15.6:0/2906139099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940095f0 con 0x7f869c0968c0 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a1d9b700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f869800d950 tx=0x7f869800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a259c700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 0x7f869c12d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86a2d9d700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 0x7f869c130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-17T00:20:21.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.456+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698009b40 con 0x7f869c0968c0 2023-11-17T00:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.457+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c0a53f0 con 0x7f869c0968c0 2023-11-17T00:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.457+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c0a5260 con 0x7f869c0968c0 2023-11-17T00:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.457+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8698009ca0 con 0x7f869c0968c0 2023-11-17T00:20:21.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.457+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800f620 con 0x7f869c0968c0 2023-11-17T00:20:21.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.458+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8684005310 con 0x7f869c0968c0 2023-11-17T00:20:21.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.467+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8698004020 con 0x7f869c0968c0 2023-11-17T00:20:21.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.470+0000 7f86937fe700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 0x7f868c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-17T00:20:21.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.470+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8458+0+0 (secure 0 0 0) 0x7f86980ccfe0 con 0x7f869c0968c0 2023-11-17T00:20:21.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.471+0000 7f86a259c700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 0x7f868c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-17T00:20:21.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.472+0000 7f86a259c700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 0x7f868c0a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f869400b5c0 tx=0x7f8694005fb0 comp rx=0 tx=0).ready entity=mgr.25469 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-17T00:20:21.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.473+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f86980cd4c0 con 0x7f869c0968c0 2023-11-17T00:20:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:21.893+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8684005180 con 0x7f869c0968c0 2023-11-17T00:20:22.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.033+0000 7f86937fe700 1 -- 172.21.15.6:0/2906139099 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) v1 ==== 178+0+0 (secure 0 0 0) 0x7f86980160d0 con 0x7f869c0968c0 2023-11-17T00:20:22.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 msgr2=0x7f868c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:22.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 0x7f868c0a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f869400b5c0 tx=0x7f8694005fb0 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 msgr2=0x7f869c12dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f869800d950 tx=0x7f869800dd10 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 shutdown_connections 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:6800/3813072549,v1:172.21.15.6:6801/3813072549] conn(0x7f868c0a1340 0x7f868c0a37f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f869c095f10 0x7f869c12d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f869c09a460 0x7f869c130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 --2- 172.21.15.6:0/2906139099 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f869c0968c0 0x7f869c12dc30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-17T00:20:22.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 >> 172.21.15.6:0/2906139099 conn(0x7f869c00b4e0 msgr2=0x7f869c094a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-17T00:20:22.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 shutdown_connections 2023-11-17T00:20:22.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-17T00:20:22.038+0000 7f86a359e700 1 -- 172.21.15.6:0/2906139099 wait complete. 2023-11-17T00:20:22.042 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' removed 2023-11-17T00:20:22.074 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:20:22.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:20:22.338 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:22 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:22.338 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:22 smithi087 ceph-mon[131182]: osdmap e106: 12 total, 12 up, 12 in 2023-11-17T00:20:22.338 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:22 smithi087 ceph-mon[131182]: from='client.? 172.21.15.6:0/2906139099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.339 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:22 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.340 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:20:22.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:22 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:22.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:22 smithi186 ceph-mon[131487]: osdmap e106: 12 total, 12 up, 12 in 2023-11-17T00:20:22.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:22 smithi186 ceph-mon[131487]: from='client.? 172.21.15.6:0/2906139099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.368 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:22 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.372 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-17T00:20:22.373 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-17T00:20:22.428 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-17T00:20:22.449 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-17T00:20:22.467 INFO:tasks.cephadm:Stopping all daemons... 2023-11-17T00:20:22.467 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-17T00:20:22.467 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a 2023-11-17T00:20:22.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:22 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:22.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:22 smithi006 ceph-mon[97840]: osdmap e106: 12 total, 12 up, 12 in 2023-11-17T00:20:22.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:22 smithi006 ceph-mon[97840]: from='client.? 172.21.15.6:0/2906139099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:22 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-17T00:20:22.973 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:22 smithi006 systemd[1]: Stopping Ceph mon.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:23 smithi087 ceph-mon[131182]: pgmap v360: 33 pgs: 1 active+clean, 15 active+clean+snaptrim, 17 active+clean+snaptrim_wait; 577 KiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:20:23.351 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:23 smithi087 ceph-mon[131182]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:23.352 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:23 smithi087 ceph-mon[131182]: osdmap e107: 12 total, 12 up, 12 in 2023-11-17T00:20:23.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:23 smithi186 ceph-mon[131487]: pgmap v360: 33 pgs: 1 active+clean, 15 active+clean+snaptrim, 17 active+clean+snaptrim_wait; 577 KiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:20:23.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:23 smithi186 ceph-mon[131487]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:23.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:23 smithi186 ceph-mon[131487]: osdmap e107: 12 total, 12 up, 12 in 2023-11-17T00:20:23.473 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:23 smithi006 ceph-mon[97840]: pgmap v360: 33 pgs: 1 active+clean, 15 active+clean+snaptrim, 17 active+clean+snaptrim_wait; 577 KiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-17T00:20:23.474 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:23 smithi006 ceph-mon[97840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-11-17T00:20:23.476 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:23 smithi006 ceph-mon[97840]: osdmap e107: 12 total, 12 up, 12 in 2023-11-17T00:20:23.477 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a[97816]: 2023-11-17T00:20:23.086+0000 7fc309686700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:23.477 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a[97816]: 2023-11-17T00:20:23.086+0000 7fc309686700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-17T00:20:25.723 INFO:journalctl@ceph.mon.a.smithi006.stdout:Nov 17 00:20:25 smithi006 bash[262369]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-a 2023-11-17T00:20:26.020 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.a.service' 2023-11-17T00:20:26.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:26.190 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-17T00:20:26.191 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-17T00:20:26.191 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.b 2023-11-17T00:20:26.536 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:26 smithi087 systemd[1]: Stopping Ceph mon.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:26.834 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-b[131158]: 2023-11-17T00:20:26.533+0000 7faebff4e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:26.835 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-b[131158]: 2023-11-17T00:20:26.533+0000 7faebff4e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-17T00:20:27.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.b.service' 2023-11-17T00:20:27.101 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:26 smithi087 bash[162708]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-b 2023-11-17T00:20:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:27 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.b.service: Succeeded. 2023-11-17T00:20:27.102 INFO:journalctl@ceph.mon.b.smithi087.stdout:Nov 17 00:20:27 smithi087 systemd[1]: Stopped Ceph mon.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:20:27.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:27.135 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-17T00:20:27.135 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-17T00:20:27.135 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.c 2023-11-17T00:20:27.615 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:27 smithi186 systemd[1]: Stopping Ceph mon.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:28.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-c[131460]: 2023-11-17T00:20:27.687+0000 7f71f6a15700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:28.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-c[131460]: 2023-11-17T00:20:27.687+0000 7f71f6a15700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-17T00:20:28.788 INFO:journalctl@ceph.mon.c.smithi186.stdout:Nov 17 00:20:28 smithi186 bash[163872]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mon-c 2023-11-17T00:20:28.795 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mon.c.service' 2023-11-17T00:20:28.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:28.847 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-17T00:20:28.847 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-17T00:20:28.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-17T00:20:29.223 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:20:28 smithi006 systemd[1]: Stopping Ceph mgr.x for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:29.680 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:20:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x[98086]: 2023-11-17T00:20:29.402+0000 7f9c690a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:29.973 INFO:journalctl@ceph.mgr.x.smithi006.stdout:Nov 17 00:20:29 smithi006 bash[262516]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-x 2023-11-17T00:20:30.293 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.x.service' 2023-11-17T00:20:30.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:30.386 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-17T00:20:30.387 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-17T00:20:30.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.y 2023-11-17T00:20:30.811 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:20:30 smithi087 systemd[1]: Stopping Ceph mgr.y for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:31.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:20:30 smithi087 bash[162939]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-y 2023-11-17T00:20:31.101 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Nov 17 00:20:31 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-17T00:20:31.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.y.service' 2023-11-17T00:20:31.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:31.232 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-17T00:20:31.233 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-17T00:20:31.233 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.z 2023-11-17T00:20:31.616 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:20:31 smithi186 systemd[1]: Stopping Ceph mgr.z for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:31.927 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:20:31 smithi186 bash[164022]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mgr-z 2023-11-17T00:20:32.109 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Nov 17 00:20:31 smithi186 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-11-17T00:20:32.113 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mgr.z.service' 2023-11-17T00:20:32.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:32.162 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-17T00:20:32.163 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-17T00:20:32.163 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.0 2023-11-17T00:20:32.723 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:20:32 smithi006 systemd[1]: Stopping Ceph osd.0 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:33.223 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:20:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0[110097]: 2023-11-17T00:20:32.776+0000 7fa1079b9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:33.223 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:20:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0[110097]: 2023-11-17T00:20:32.776+0000 7fa1079b9700 -1 osd.0 107 *** Got signal Terminated *** 2023-11-17T00:20:33.224 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:20:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0[110097]: 2023-11-17T00:20:32.776+0000 7fa1079b9700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:20:39.223 INFO:journalctl@ceph.osd.0.smithi006.stdout:Nov 17 00:20:38 smithi006 bash[262662]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-0 2023-11-17T00:20:41.996 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.0.service' 2023-11-17T00:20:42.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:42.090 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-17T00:20:42.091 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-17T00:20:42.091 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.1 2023-11-17T00:20:42.459 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:20:42 smithi087 systemd[1]: Stopping Ceph osd.1 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:42.851 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:20:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:20:42.456+0000 7fdf0c4d3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:42.851 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:20:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:20:42.456+0000 7fdf0c4d3700 -1 osd.1 107 *** Got signal Terminated *** 2023-11-17T00:20:42.851 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:20:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1[136228]: 2023-11-17T00:20:42.456+0000 7fdf0c4d3700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:20:48.351 INFO:journalctl@ceph.osd.1.smithi087.stdout:Nov 17 00:20:48 smithi087 bash[163083]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-1 2023-11-17T00:20:49.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.1.service' 2023-11-17T00:20:49.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:49.489 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-17T00:20:49.505 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-17T00:20:49.505 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.2 2023-11-17T00:20:49.861 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:20:49 smithi186 systemd[1]: Stopping Ceph osd.2 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:50.115 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:20:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:20:49.858+0000 7ff6653a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:50.116 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:20:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:20:49.858+0000 7ff6653a8700 -1 osd.2 107 *** Got signal Terminated *** 2023-11-17T00:20:50.116 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:20:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2[137322]: 2023-11-17T00:20:49.858+0000 7ff6653a8700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:20:55.615 INFO:journalctl@ceph.osd.2.smithi186.stdout:Nov 17 00:20:55 smithi186 bash[164166]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-2 2023-11-17T00:20:56.884 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.2.service' 2023-11-17T00:20:56.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:20:56.924 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-17T00:20:56.924 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-17T00:20:56.924 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.3 2023-11-17T00:20:57.473 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:20:57 smithi006 systemd[1]: Stopping Ceph osd.3 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:20:58.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:20:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:20:57.840+0000 7fa92c9a7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:20:58.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:20:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:20:57.840+0000 7fa92c9a7700 -1 osd.3 107 *** Got signal Terminated *** 2023-11-17T00:20:58.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:20:57 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:20:57.840+0000 7fa92c9a7700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:20:59.101 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:20:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:20:58.643+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:00.101 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:20:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:20:59.611+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:00.224 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:20:59 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:20:59.869+0000 7fa928f9d700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.139886+0000 front 2023-11-17T00:20:36.139961+0000 (oldest deadline 2023-11-17T00:20:59.040098+0000) 2023-11-17T00:21:00.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:20:59 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:20:59.749+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:00.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:20:59 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:20:59.875+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:00.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:00 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:00.569+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:01.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:21:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:21:00.830+0000 7fa928f9d700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.139886+0000 front 2023-11-17T00:20:36.139961+0000 (oldest deadline 2023-11-17T00:20:59.040098+0000) 2023-11-17T00:21:01.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:00.782+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:01.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:00 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:00.874+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:01.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:01 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:01.536+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:02.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:21:01 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3[115556]: 2023-11-17T00:21:01.859+0000 7fa928f9d700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.139886+0000 front 2023-11-17T00:20:36.139961+0000 (oldest deadline 2023-11-17T00:20:59.040098+0000) 2023-11-17T00:21:02.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:01 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:01.804+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:02.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:01 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:01.842+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:02.365 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:02.006+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:02.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:02 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:02.521+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:03.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:02 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:02.853+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:03.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:02 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:02.879+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:03.365 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:03.001+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:03.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:03.216+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:03.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:03 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:03.478+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:04.223 INFO:journalctl@ceph.osd.3.smithi006.stdout:Nov 17 00:21:03 smithi006 bash[262935]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-3 2023-11-17T00:21:04.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:03 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:03.848+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:04.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:03 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:03.903+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:04.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:04.007+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:04.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:04.202+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:04.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:04 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:04.521+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:05.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:04 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:04.825+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:05.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:04 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:04.886+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:05.200+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:05.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:05.001+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:05.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:05 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:05.520+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:05.967 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.3.service' 2023-11-17T00:21:06.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:06.060 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-17T00:21:06.060 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-17T00:21:06.060 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.4 2023-11-17T00:21:06.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:05 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:05.824+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:06.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:05 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:05.856+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:06.352 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:06 smithi087 systemd[1]: Stopping Ceph osd.4 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:06.365 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:06.046+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:06.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:06.244+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:06.853 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:06 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:06.532+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:06.854 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:06 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:21:06.428+0000 7f04ea82a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:06.854 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:06 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:21:06.428+0000 7f04ea82a700 -1 osd.4 107 *** Got signal Terminated *** 2023-11-17T00:21:06.854 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:06 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:21:06.428+0000 7f04ea82a700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:07.228 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:06 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:06.835+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:07.228 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:06 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:06.905+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:07.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:07.077+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:07.264+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:07.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:07 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:07.573+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:08.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:07 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:07.822+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:08.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:07 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:07.951+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:08.365 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:08.038+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:08.257+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:08.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:08 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:08.538+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:09.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:08 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:08.872+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:09.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:08 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:08.951+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:09.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:08 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:08.871+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:09.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:09.061+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:09.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:09.292+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:09.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:09 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:09.514+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:10.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:09 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:09.848+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:10.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:09 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:09.927+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:10.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:09 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:09.928+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:10.326 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:10 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:10.039+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:10.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:09 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:09.888+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:10.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:10 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:10.325+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:10.841 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:10 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:10.526+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:10.841 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:10 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:10.526+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:11.101 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:10 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4[141759]: 2023-11-17T00:21:10.839+0000 7f04e6e20700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.908254+0000 front 2023-11-17T00:20:46.908281+0000 (oldest deadline 2023-11-17T00:21:09.808446+0000) 2023-11-17T00:21:11.102 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:10 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:10.876+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:11.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:10.835+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:11.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:10.927+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:11.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:10 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:10.927+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:11.362 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:11 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:11.079+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:11.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:11 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:11.361+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:11.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:11 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:11.514+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:11.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:11 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:11.514+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:12.211 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:11 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:11.895+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:12.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:11.801+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:12.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:11.950+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:12.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:11 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:11.950+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:12.325 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:12.052+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:12.531 INFO:journalctl@ceph.osd.4.smithi087.stdout:Nov 17 00:21:12 smithi087 bash[163351]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-4 2023-11-17T00:21:12.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:12.324+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:12.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:12.324+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:12.841 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:12 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:12.530+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:12.841 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:12 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:12.530+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:13.039 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:12 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:12.693+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:13.101 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:12 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:12.862+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:13.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:12.775+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:13.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:12.775+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:13.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:12.908+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:13.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:12.908+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:13.360 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:13.038+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:13.360 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:13.038+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:13.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.4.service' 2023-11-17T00:21:13.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:13.608 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-17T00:21:13.609 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-17T00:21:13.609 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.5 2023-11-17T00:21:13.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:13.359+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:13.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:13.359+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:13.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:13 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:13.577+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:13.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:13 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:13.577+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:13.973 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 systemd[1]: Stopping Ceph osd.5 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:13.973 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:13.694+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:14.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:13.816+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:14.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:13.816+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:14.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:13.859+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:14.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:13 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:13.859+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:14.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:13 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:13.853+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:14.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:13.972+0000 7f79ae20a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:14.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:13.972+0000 7f79ae20a700 -1 osd.5 107 *** Got signal Terminated *** 2023-11-17T00:21:14.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:13 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:13.972+0000 7f79ae20a700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:14.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:14.080+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:14.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:14.080+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:14.691 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:14.393+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:14.692 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:14.393+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:14.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:14.550+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:14.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:14.550+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:15.114 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:14 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:14.689+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:15.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:14.837+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:15.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:14.837+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:15.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:14.853+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:15.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:14 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:14.853+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:15.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:14 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:14.865+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:15.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:15.112+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:15.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:15.112+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:15.701 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:15.405+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:15.701 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:15.406+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:15.843 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:15.524+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:15.843 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:15.524+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:16.101 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:15 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:15.842+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:16.115 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:15 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:15.699+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:16.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:15.880+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:16.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:15.880+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:16.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:15.850+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:16.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:15.850+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:16.424 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:16.120+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:16.425 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:16.120+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:16.748 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:16.422+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:16.749 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:16.422+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:16.817 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:16.483+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:16.817 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:16.483+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:17.073 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:16 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:16.746+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:17.102 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:16 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:16.815+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:17.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:16.922+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:17.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:16.922+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:17.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:16.922+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:17.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:16.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:17.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:16.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:17.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:16 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:16.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:17.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:17.071+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:17.366 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:17.071+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:17.712 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:17.380+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:17.712 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:17.380+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:17.712 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:17.380+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:17.845 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:17.488+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:17.845 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:17.488+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:17.846 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:17.488+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:18.084 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:17 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:17.710+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:18.101 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:17.843+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:18.101 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:17 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:17.843+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:18.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:17.955+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:18.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:17.955+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:18.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:17.955+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:18.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:17.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:18.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:17.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:18.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:17 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:17.896+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:18.359 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:18.082+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:33.108082+0000 front 2023-11-17T00:20:33.107925+0000 (oldest deadline 2023-11-17T00:21:01.308605+0000) 2023-11-17T00:21:18.359 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5[142712]: 2023-11-17T00:21:18.082+0000 7f79aa800700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.110253+0000 front 2023-11-17T00:20:47.110332+0000 (oldest deadline 2023-11-17T00:21:12.310730+0000) 2023-11-17T00:21:18.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:18.357+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:18.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:18.357+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:18.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:18.357+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:18.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:18.440+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:18.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:18.440+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:18.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:18.440+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:19.120 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:18.696+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:19.131 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:18 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:18.696+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:19.223 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:18.911+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:19.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:18.911+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:19.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:18.911+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:19.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:18.935+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:19.225 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:18.935+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:19.225 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:18 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:18.935+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:19.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:18.894+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:19.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:18 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:18.894+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:19.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:19.330+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:19.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:19.330+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:19.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:19.330+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:19.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:19.429+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:19.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:19.429+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:19.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:19.429+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:20.116 INFO:journalctl@ceph.osd.5.smithi186.stdout:Nov 17 00:21:19 smithi186 bash[164439]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-5 2023-11-17T00:21:20.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:19.739+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:20.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:19 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:19.739+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:20.229 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:19.932+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:20.254 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:19.932+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:20.255 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:19.932+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:20.255 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:19.922+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:20.255 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:19.922+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:20.255 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:19 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:19.922+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:20.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:19.911+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:20.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:19 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:19.911+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:20.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:20 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:20.316+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:20.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:20 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:20.316+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:20.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:20 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:20.316+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:20.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:20.417+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:20.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:20.417+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:20.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:20.417+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:21.000 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:20 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:20.706+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:21.001 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:20 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:20.706+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:21.006 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.5.service' 2023-11-17T00:21:21.087 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:21.087 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-17T00:21:21.087 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-17T00:21:21.088 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.6 2023-11-17T00:21:21.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:20.959+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:21.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:20.959+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:21.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:20.959+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:21.224 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:20.968+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:21.225 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:20.968+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:21.225 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:20 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:20.968+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:21.225 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 systemd[1]: Stopping Ceph osd.6 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:21.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:20.866+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:21.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:20 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:20.866+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:21.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:21 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:21.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:21.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:21 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:21.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:21.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:21 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:21.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:21.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:21 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:21.425+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:21.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:21 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:21.425+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:21.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:21 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:21.425+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:21.973 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.646+0000 7f2021e34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:21.973 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.646+0000 7f2021e34700 -1 osd.6 107 *** Got signal Terminated *** 2023-11-17T00:21:21.973 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.646+0000 7f2021e34700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:22.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:21 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:21.715+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:22.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:21 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:21.715+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:22.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:21 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:21.914+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:22.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:21 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:21.914+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:22.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:21.998+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:22.623 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:21.998+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:22.623 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:21.998+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:22.623 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.993+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:22.623 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.993+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:22.623 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:21.993+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:22.624 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:22 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:22.309+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:22.624 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:22 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:22.309+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:22.624 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:22 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:22.309+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:22.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:22 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:22.418+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:22.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:22 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:22.418+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:22.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:22 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:22.418+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:23.115 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:22 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:22.721+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:23.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:22 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:22.721+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:23.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:22 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:22.904+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:23.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:22 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:22.904+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:23.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:23.044+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:23.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:23.044+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:23.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:23.044+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:23.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:23.040+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:23.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:23.040+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:23.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:23 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:23.040+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:23.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:23.260+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:23.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:23.260+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:23.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:23.260+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:23.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:23 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:23.409+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:23.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:23 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:23.409+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:23.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:23 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:23.409+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:24.115 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:23.760+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:24.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:23.760+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:24.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:23 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:23.760+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:24.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:23 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:23.859+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:24.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:23 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:23.859+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:24.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:24.030+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:24.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:24.030+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:24.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:24.030+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:24.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:24.088+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:24.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:24.088+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:24.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:24.088+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:24.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:24.088+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.149848+0000 front 2023-11-17T00:20:58.149825+0000 (oldest deadline 2023-11-17T00:21:24.050124+0000) 2023-11-17T00:21:24.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:24.226+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:24.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:24.226+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:24.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:24.226+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:24.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:24 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:24.375+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:24.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:24 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:24.375+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:24.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:24 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:24.375+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:25.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:24.730+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:25.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:24.730+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:25.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:24 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:24.730+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:25.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:24 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:24.899+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:25.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:24 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:24.899+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:25.473 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:25.069+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:25.473 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:25.069+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:25.069+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:25.069+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.149848+0000 front 2023-11-17T00:20:58.149825+0000 (oldest deadline 2023-11-17T00:21:24.050124+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:25.048+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:25.048+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:25.048+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:25.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:25 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:25.048+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:25.618 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:25.262+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:25.618 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:25.262+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:25.619 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:25.262+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:25.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:25.368+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:25.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:25.368+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:25.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:25.368+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:26.115 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:25.773+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:26.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:25.773+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:26.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:25 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:25.773+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:26.330 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:25.921+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:26.330 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:25 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:25.921+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:26.473 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:26.108+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:35.643373+0000 front 2023-11-17T00:20:35.643773+0000 (oldest deadline 2023-11-17T00:20:59.143312+0000) 2023-11-17T00:21:26.473 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:26.109+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.946881+0000 front 2023-11-17T00:20:45.946755+0000 (oldest deadline 2023-11-17T00:21:11.846802+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:26.109+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:51.847767+0000 front 2023-11-17T00:20:51.847797+0000 (oldest deadline 2023-11-17T00:21:15.947771+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6[121119]: 2023-11-17T00:21:26.109+0000 7f201cc27700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.149848+0000 front 2023-11-17T00:20:58.149825+0000 (oldest deadline 2023-11-17T00:21:24.050124+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:26.098+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:26.098+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:26.098+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:26.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:26 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:26.098+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:26.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:26.328+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:26.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:26.328+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:26.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:26.328+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:26.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:26.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:26.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:26.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:26.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:26.290+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:27.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:26.821+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:27.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:26.821+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:27.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:26 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:26.821+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:27.327 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:26.928+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:27.328 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:26.928+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:27.328 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:26 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:26.928+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:27.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:27.124+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:27.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:27.124+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:27.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:27.124+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:27.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:27 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:27.124+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:27.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:27.325+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:27.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:27.325+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:27.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:27.325+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:27.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:27.325+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:27.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:27.337+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:27.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:27.337+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:27.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:27.337+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:28.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:27.809+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:28.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:27.809+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:28.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:27 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:27.809+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:28.142 INFO:journalctl@ceph.osd.6.smithi006.stdout:Nov 17 00:21:27 smithi006 bash[263213]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-6 2023-11-17T00:21:28.319 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:27.949+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:28.320 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:27.949+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:28.320 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:27 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:27.949+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:28.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:28.141+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:28.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:28.141+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:28.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:28.141+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:28.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:28 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:28.141+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:28.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:28.317+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:28.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:28.317+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:28.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:28.317+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:28.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:28.317+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:28.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:28.298+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:28.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:28.298+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:28.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:28.299+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:29.115 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:28.823+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:29.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:28.823+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:29.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:28 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:28.823+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:29.346 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:28.914+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:29.346 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:28.914+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:29.347 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:28 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:28.914+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:29.418 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:29.123+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:29.418 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:29.123+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:29.418 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:29.123+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:29.419 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:29 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:29.123+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:29.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:29.341+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:29.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:29.341+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:29.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:29.341+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:29.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:29.341+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:29.615 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:29.275+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:29.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:29.275+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:29.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:29.275+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:30.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:29.859+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:30.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:29.859+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:30.116 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:29 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:29.859+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:30.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:30 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:30.102+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:30.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:30 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:30.102+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:30.223 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:30 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:30.102+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:30.224 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:30 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:30.102+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:30.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:29.954+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:30.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:29.954+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:30.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:29.954+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:30.458 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.6.service' 2023-11-17T00:21:30.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:30.555 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-17T00:21:30.555 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-17T00:21:30.555 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.7 2023-11-17T00:21:30.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:30.240+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:30.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:30.240+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:30.617 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:30.240+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:30.851 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 systemd[1]: Stopping Ceph osd.7 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:30.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:30.359+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:30.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:30.359+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:30.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:30.359+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:30.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:30.359+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:31.194 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:30.884+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:31.194 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:30.884+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:31.194 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:30 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:30.884+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:31.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.908+0000 7f3bd0ce8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:31.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.908+0000 7f3bd0ce8700 -1 osd.7 107 *** Got signal Terminated *** 2023-11-17T00:21:31.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.908+0000 7f3bd0ce8700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:31.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.978+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:31.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.978+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:31.353 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:30 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:30.978+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:31.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:31.059+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:31.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:31.059+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:31.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:31.059+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:31.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:31 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:31.059+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:31.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:31.192+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:31.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:31.192+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:31.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:31.192+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:31.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:31 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:31.383+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:31.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:31 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:31.383+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:31.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:31 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:31.383+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:31.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:31 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:31.383+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:32.169 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:31.881+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:32.169 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:31.881+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:32.169 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:31 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:31.881+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:32.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.011+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:32.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.011+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:32.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.011+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:32.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:32.058+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:32.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:32.058+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:32.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:32.058+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:32.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:32 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:32.058+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:32.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:32.167+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:32.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:32.167+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:32.616 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:32.167+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:32.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:32.350+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:32.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:32.350+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:32.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:32.350+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:32.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:32.350+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:33.351 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.965+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:33.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.965+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:33.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.965+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:33.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:32 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:32.965+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:07.169955+0000 front 2023-11-17T00:21:07.169934+0000 (oldest deadline 2023-11-17T00:21:32.470142+0000) 2023-11-17T00:21:33.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:32.929+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:33.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:32.929+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:33.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:32 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:32.929+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:33.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:33.141+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:33.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:33.141+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:33.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:33.141+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:33.368 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:33.141+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:33.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:33.104+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:33.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:33.104+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:33.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:33.104+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:33.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:33.104+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:33.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:33 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:33.104+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:33.862 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:33.393+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:33.865 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:33.393+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:33.865 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:33.393+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:33.866 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:33.393+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:34.352 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:33.942+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:34.353 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:33.942+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:34.353 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:33.942+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:34.354 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:33.942+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:07.169955+0000 front 2023-11-17T00:21:07.169934+0000 (oldest deadline 2023-11-17T00:21:32.470142+0000) 2023-11-17T00:21:34.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:33.948+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:34.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:33.948+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:34.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:33 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:33.948+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:34.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:34.094+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:34.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:34.094+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:34.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:34.094+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:34.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:34.094+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:34.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:34 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:34.153+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:34.484 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:34 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:34.153+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:34.484 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:34 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:34.153+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:34.485 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:34 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:34.153+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:34.485 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:34 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:34.153+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:34.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:34.386+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:34.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:34.386+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:34.851 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:34.386+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:34.852 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:34.386+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:35.344 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:34.910+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:46.965205+0000 front 2023-11-17T00:20:46.965229+0000 (oldest deadline 2023-11-17T00:21:08.065402+0000) 2023-11-17T00:21:35.345 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:34.910+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.166656+0000 front 2023-11-17T00:20:52.166573+0000 (oldest deadline 2023-11-17T00:21:17.466762+0000) 2023-11-17T00:21:35.345 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:34.910+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:21:02.668242+0000 front 2023-11-17T00:21:02.668285+0000 (oldest deadline 2023-11-17T00:21:26.069067+0000) 2023-11-17T00:21:35.345 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:34 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7[147236]: 2023-11-17T00:21:34.910+0000 7f3bcb2da700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:07.169955+0000 front 2023-11-17T00:21:07.169934+0000 (oldest deadline 2023-11-17T00:21:32.470142+0000) 2023-11-17T00:21:35.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:34.955+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:35.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:34.955+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:35.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:34 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:34.955+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:35.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:35.060+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:35.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:35.060+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:35.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:35.060+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:35.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:35.060+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:35.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:35.183+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:35.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:35.183+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:35.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:35.183+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:35.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:35.183+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:35.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:35 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:35.183+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:35.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:35.343+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:35.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:35.343+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:35.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:35.343+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:35.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:35.343+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:36.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:35.989+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:36.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:35.989+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:36.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:35.989+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:36.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:35 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:35.989+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:36.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:36.068+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:36.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:36.068+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:36.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:36.068+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:36.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:36.068+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:36.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:36 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:36.221+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:36.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:36 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:36.221+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:36.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:36 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:36.221+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:36.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:36 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:36.221+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:36.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:36 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:36.221+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:36.697 INFO:journalctl@ceph.osd.7.smithi087.stdout:Nov 17 00:21:36 smithi087 bash[163621]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-7 2023-11-17T00:21:36.697 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:36 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:36.352+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:36.698 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:36 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:36.352+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:36.698 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:36 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:36.352+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:36.698 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:36 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:36.352+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:36.698 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:36 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:36.352+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:37.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:37.041+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:37.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:37.041+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:37.041+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:37.041+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:36.959+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:36.959+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:36.959+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:37.370 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:36 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:36.959+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:37.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:37 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:37.311+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:37.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:37 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:37.311+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:37.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:37 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:37.311+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:37.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:37 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:37.311+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:37.603 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:37 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:37.311+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:37.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:37.223+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:37.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:37.223+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:37.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:37.223+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:37.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:37.223+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:37.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:37 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:37.223+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:38.050 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.7.service' 2023-11-17T00:21:38.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:38.089 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-17T00:21:38.089 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-17T00:21:38.089 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.8 2023-11-17T00:21:38.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:37.994+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:38.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:37.994+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:38.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:37.994+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:38.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:37 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:37.994+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:38.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:38 smithi186 systemd[1]: Stopping Ceph osd.8 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:38.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:38.049+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:38.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:38.049+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:38.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:38.049+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:38.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:38.049+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:38.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:38 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:38.314+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:38.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:38 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:38.314+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:38.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:38 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:38.314+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:38.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:38 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:38.314+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:38.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:38 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:38.314+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:38.725 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:38.256+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:38.741 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:38.256+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:38.741 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:38.256+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:38.741 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:38.256+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:38.741 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:38 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:38.256+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:38.866 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:38.443+0000 7f42108f4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:38.866 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:38.443+0000 7f42108f4700 -1 osd.8 107 *** Got signal Terminated *** 2023-11-17T00:21:38.866 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:38 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:38.443+0000 7f42108f4700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:39.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:39.063+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:39.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:39.063+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:39.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:39.064+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:39.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:39.064+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:39.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:39.007+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:39.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:39.007+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:39.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:39.007+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:39.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:39 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:39.007+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:39.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:39 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:39.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:39.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:39 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:39.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:39.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:39 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:39.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:39.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:39 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:39.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:39.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:39 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:39.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:39.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:39.268+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:39.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:39.268+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:39.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:39.268+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:39.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:39.268+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:39.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:39 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:39.268+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:40.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:40.035+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:40.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:40.035+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:40.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:40.035+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:40.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:40.035+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:40.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:40.035+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:15.311291+0000 front 2023-11-17T00:21:15.311321+0000 (oldest deadline 2023-11-17T00:21:40.011413+0000) 2023-11-17T00:21:40.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:40.071+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:40.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:40.071+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:40.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:40.071+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:40.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:40 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:40.071+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:40.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:40.232+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:40.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:40.232+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:40.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:40.232+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:40.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:40.232+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:40.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:40.232+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:40.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:40.260+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:40.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:40.260+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:40.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:40.260+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:40.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:40.260+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:40.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:40 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:40.260+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:41.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:41.078+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:41.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:41.078+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:41.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:41.078+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:41.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:41.078+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:41.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:41.078+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:15.311291+0000 front 2023-11-17T00:21:15.311321+0000 (oldest deadline 2023-11-17T00:21:40.011413+0000) 2023-11-17T00:21:41.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:41.035+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:41.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:41.035+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:41.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:41.035+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:41.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:41.035+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:41.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:41 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:41.035+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:41.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:41 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:41.277+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:41.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:41 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:41.277+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:41.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:41 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:41.277+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:41.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:41 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:41.277+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:41.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:41 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:41.277+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:41.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:41.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:41.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:41.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:41.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:41.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:41 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:41.276+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:42.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:42.119+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:42.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:42.119+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:42.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:42.119+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:42.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:42.119+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:42.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:42.119+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:15.311291+0000 front 2023-11-17T00:21:15.311321+0000 (oldest deadline 2023-11-17T00:21:40.011413+0000) 2023-11-17T00:21:42.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.002+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:42.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.002+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:42.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.002+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:42.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.002+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:42.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.002+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:42.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:42.272+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:42.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:42.272+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:42.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:42.272+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:42.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:42.272+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:42.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:42 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:42.272+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:42.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:42.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:42.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:42.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:42.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:42.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:42 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:42.279+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:43.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:43.108+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.406256+0000 front 2023-11-17T00:20:47.406201+0000 (oldest deadline 2023-11-17T00:21:12.106229+0000) 2023-11-17T00:21:43.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:43.108+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.407362+0000 front 2023-11-17T00:20:54.407385+0000 (oldest deadline 2023-11-17T00:21:18.407965+0000) 2023-11-17T00:21:43.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:43.108+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:58.408476+0000 front 2023-11-17T00:20:58.408405+0000 (oldest deadline 2023-11-17T00:21:23.708467+0000) 2023-11-17T00:21:43.366 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:43.108+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.611054+0000 front 2023-11-17T00:21:10.611135+0000 (oldest deadline 2023-11-17T00:21:35.310957+0000) 2023-11-17T00:21:43.367 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8[148112]: 2023-11-17T00:21:43.108+0000 7f42096e3700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:15.311291+0000 front 2023-11-17T00:21:15.311321+0000 (oldest deadline 2023-11-17T00:21:40.011413+0000) 2023-11-17T00:21:43.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:43.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:43.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:43.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:43.368 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:42.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:43.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.289+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:43.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.289+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:43.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.289+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:43.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.289+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:43.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.290+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:43.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:43 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:43.290+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:43.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:43.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:43.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:43.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:43.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:43.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:43 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:43.246+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:44.268 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:43.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:44.268 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:43.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:44.269 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:43.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:44.269 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:43.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:44.269 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:43 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:43.981+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:44.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:44.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:44.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:44.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:44.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:44.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:44 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:44.284+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:44.616 INFO:journalctl@ceph.osd.8.smithi186.stdout:Nov 17 00:21:44 smithi186 bash[164699]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-8 2023-11-17T00:21:44.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:44.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:44.723 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:44.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:44.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:44.724 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:44 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:44.245+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:45.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:45.018+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:45.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:45.018+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:45.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:45.018+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:45.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:45.018+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:45.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:45 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:45.018+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:45.586 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.8.service' 2023-11-17T00:21:45.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:45.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:45.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:45.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:45.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:45.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:45 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:45.304+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:45.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:45.626 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-17T00:21:45.626 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-17T00:21:45.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.9 2023-11-17T00:21:45.697 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:45.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:45.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:45.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:45.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:45.698 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:45.282+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:45.973 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:45 smithi006 systemd[1]: Stopping Ceph osd.9 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:21:46.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:46.032+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:46.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:46.032+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:46.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:46.032+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:46.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:46.032+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:46.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:46 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:46.032+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:46.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.175+0000 7fe2edb87700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:21:46.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.175+0000 7fe2edb87700 -1 osd.9 107 *** Got signal Terminated *** 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.175+0000 7fe2edb87700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:46.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:46.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:46 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:46.244+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:46.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:46.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:46.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:46.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:46.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:46.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:46 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:46.270+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:47.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:47.048+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:47.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:47.048+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:47.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:47.048+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:47.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:47.048+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:47.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:47.048+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:47.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:47.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:47.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:47.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:47.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:47.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:47 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:47.218+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:47.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:47.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:47.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:47.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:47.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:47.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:47 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:47.266+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:48.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:48.088+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:48.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:48.088+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:48.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:48.088+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:48.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:48.088+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:48.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:48 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:48.088+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:48.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:48.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:48.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:48.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:48.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:48.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:48 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:48.204+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:48.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:48.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:48 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:48.297+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:49.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:49.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:49.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:49.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:49.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:49.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:49.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:49.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:49.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:49 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:49.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:49.475 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:49.476 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:49.476 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:49.476 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:49.476 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:49.476 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:49 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:49.217+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:49.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:49.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:49 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:49.313+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:50.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:50.059+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:50.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:50.059+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:50.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:50.059+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:50.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:50.059+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:50.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:50 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:50.059+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:50.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:50.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:50.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:50.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:50.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:50.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:50 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:50.172+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:50.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:50.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:50.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:50.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:50.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:50.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:50.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:50 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:50.305+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:51.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:51.082+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:51.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:51.082+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:51.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:51.082+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:51.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:51.082+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:51.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:51 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:51.082+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:51.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.256864+0000 front 2023-11-17T00:20:36.257152+0000 (oldest deadline 2023-11-17T00:20:59.757080+0000) 2023-11-17T00:21:51.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:45.558749+0000 front 2023-11-17T00:20:45.558717+0000 (oldest deadline 2023-11-17T00:21:09.659042+0000) 2023-11-17T00:21:51.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:54.360260+0000 front 2023-11-17T00:20:54.360500+0000 (oldest deadline 2023-11-17T00:21:16.060445+0000) 2023-11-17T00:21:51.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:59.561273+0000 front 2023-11-17T00:20:59.561316+0000 (oldest deadline 2023-11-17T00:21:24.861432+0000) 2023-11-17T00:21:51.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.162721+0000 front 2023-11-17T00:21:10.162882+0000 (oldest deadline 2023-11-17T00:21:32.963833+0000) 2023-11-17T00:21:51.474 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:51 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9[126754]: 2023-11-17T00:21:51.160+0000 7fe2e7978700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.365796+0000 front 2023-11-17T00:21:16.365641+0000 (oldest deadline 2023-11-17T00:21:40.465751+0000) 2023-11-17T00:21:51.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:51.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:51.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:51.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:51.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:51.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:51.603 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:51 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:51.296+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:52.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:52.044+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:52.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:52.044+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:52.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:52.044+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:52.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:52.044+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:52.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:52.044+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:52.473 INFO:journalctl@ceph.osd.9.smithi006.stdout:Nov 17 00:21:52 smithi006 bash[263482]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-9 2023-11-17T00:21:52.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:52.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:52.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:52.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:52.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:52.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:52.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:52 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:52.301+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:21:53.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:53 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:53.079+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:21:53.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:21:53.602 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:53 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:53.329+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:21:54.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.9.service' 2023-11-17T00:21:54.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:21:54.390 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-17T00:21:54.390 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-17T00:21:54.390 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.10 2023-11-17T00:21:54.601 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:03.990 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:03.991 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:03.991 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:03.992 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.030 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.030 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.339+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.030 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 systemd[1]: Stopping Ceph osd.10 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:04.032 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.738+0000 7fa641b9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:04.033 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.738+0000 7fa641b9e700 -1 osd.10 107 *** Got signal Terminated *** 2023-11-17T00:22:04.033 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:54 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:54.738+0000 7fa641b9e700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:22:04.139 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:04.139 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:04.139 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:04.140 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:04.141 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.141 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.170 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:55 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:55.327+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.171 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:04.171 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:04.171 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:04.171 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:04.172 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.172 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.172 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:56 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:56.293+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.172 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:04.172 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:04.173 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:04.173 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:04.173 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.173 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.173 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:57 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:57.319+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.174 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:04.174 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:58 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:58.286+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.172665+0000 front 2023-11-17T00:20:36.172631+0000 (oldest deadline 2023-11-17T00:20:58.472427+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:44.373469+0000 front 2023-11-17T00:20:44.373510+0000 (oldest deadline 2023-11-17T00:21:09.673555+0000) 2023-11-17T00:22:04.203 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:49.674137+0000 front 2023-11-17T00:20:49.673926+0000 (oldest deadline 2023-11-17T00:21:17.274512+0000) 2023-11-17T00:22:04.204 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6814 osd.3 since back 2023-11-17T00:20:57.274947+0000 front 2023-11-17T00:20:57.275072+0000 (oldest deadline 2023-11-17T00:21:26.675466+0000) 2023-11-17T00:22:04.204 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.676968+0000 front 2023-11-17T00:21:10.677009+0000 (oldest deadline 2023-11-17T00:21:35.977156+0000) 2023-11-17T00:22:04.204 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:18.878043+0000 front 2023-11-17T00:21:18.878123+0000 (oldest deadline 2023-11-17T00:21:42.377993+0000) 2023-11-17T00:22:04.204 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:21:59 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10[152791]: 2023-11-17T00:21:59.248+0000 7fa63a98d700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:25.279106+0000 front 2023-11-17T00:21:25.279243+0000 (oldest deadline 2023-11-17T00:21:48.179026+0000) 2023-11-17T00:22:04.204 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:22:00 smithi087 bash[163880]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-10 2023-11-17T00:22:04.206 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.206 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.206 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.206 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.206 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.207 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:54 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:54.118+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.246 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.246 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.246 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.246 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.246 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:55 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:55.104+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:56 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:56.085+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.247 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:57 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:57.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.248 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:58.095+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.249 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.250 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.304 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.304 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:21:59 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:21:59.119+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:00 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:00.113+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.305 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:01 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:01.074+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.306 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:02 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:02.039+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.307 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.308 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.308 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.308 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.308 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:03 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:03.012+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.309 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.10.service' 2023-11-17T00:22:04.442 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:22:01 smithi087 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.10.service: Succeeded. 2023-11-17T00:22:04.452 INFO:journalctl@ceph.osd.10.smithi087.stdout:Nov 17 00:22:01 smithi087 systemd[1]: Stopped Ceph osd.10 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:04.453 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.051+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:04.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:04.500 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-17T00:22:04.500 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-17T00:22:04.500 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.11 2023-11-17T00:22:04.866 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 systemd[1]: Stopping Ceph osd.11 for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.864+0000 7f7325fb3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.864+0000 7f7325fb3700 -1 osd.11 107 *** Got signal Terminated *** 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:04 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:04.864+0000 7f7325fb3700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:05.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:05.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:05.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:05.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:05.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:05 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:05.024+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2023-11-17T00:21:42.011060+0000 front 2023-11-17T00:21:42.011148+0000 (oldest deadline 2023-11-17T00:22:04.911113+0000) 2023-11-17T00:22:06.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:06.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:06.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:06.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:06.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:06.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:06.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:06.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:06 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:06.027+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2023-11-17T00:21:42.011060+0000 front 2023-11-17T00:21:42.011148+0000 (oldest deadline 2023-11-17T00:22:04.911113+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:07.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:07.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:07.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:07 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:07.050+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2023-11-17T00:21:42.011060+0000 front 2023-11-17T00:21:42.011148+0000 (oldest deadline 2023-11-17T00:22:04.911113+0000) 2023-11-17T00:22:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:08.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:08.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:08.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:08.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:08 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:08.054+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2023-11-17T00:21:42.011060+0000 front 2023-11-17T00:21:42.011148+0000 (oldest deadline 2023-11-17T00:22:04.911113+0000) 2023-11-17T00:22:09.366 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.091+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2023-11-17T00:20:36.800797+0000 front 2023-11-17T00:20:36.800570+0000 (oldest deadline 2023-11-17T00:21:02.700666+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6804 osd.1 since back 2023-11-17T00:20:47.302054+0000 front 2023-11-17T00:20:47.301959+0000 (oldest deadline 2023-11-17T00:21:12.002125+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-11-17T00:20:52.002573+0000 front 2023-11-17T00:20:52.002561+0000 (oldest deadline 2023-11-17T00:21:17.302618+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6812 osd.4 since back 2023-11-17T00:21:10.005642+0000 front 2023-11-17T00:21:10.005670+0000 (oldest deadline 2023-11-17T00:21:32.905787+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6812 osd.5 since back 2023-11-17T00:21:16.406717+0000 front 2023-11-17T00:21:16.406734+0000 (oldest deadline 2023-11-17T00:21:40.506839+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.6:6822 osd.6 since back 2023-11-17T00:21:26.407880+0000 front 2023-11-17T00:21:26.407869+0000 (oldest deadline 2023-11-17T00:21:52.307935+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.87:6820 osd.7 since back 2023-11-17T00:21:33.909421+0000 front 2023-11-17T00:21:33.909386+0000 (oldest deadline 2023-11-17T00:21:56.809375+0000) 2023-11-17T00:22:09.367 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:09 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11[153664]: 2023-11-17T00:22:09.092+0000 7f73205a5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.186:6820 osd.8 since back 2023-11-17T00:21:42.011060+0000 front 2023-11-17T00:21:42.011148+0000 (oldest deadline 2023-11-17T00:22:04.911113+0000) 2023-11-17T00:22:11.115 INFO:journalctl@ceph.osd.11.smithi186.stdout:Nov 17 00:22:10 smithi186 bash[164972]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-osd-11 2023-11-17T00:22:12.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@osd.11.service' 2023-11-17T00:22:12.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:12.281 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-17T00:22:12.282 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-17T00:22:12.282 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.a 2023-11-17T00:22:12.723 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:22:12 smithi006 systemd[1]: Stopping Ceph mds.a for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:13.223 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:22:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-a[131857]: 2023-11-17T00:22:12.830+0000 7fc945357700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:13.223 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:22:12 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-a[131857]: 2023-11-17T00:22:12.830+0000 7fc945357700 -1 mds.a *** got signal Terminated *** 2023-11-17T00:22:14.973 INFO:journalctl@ceph.mds.a.smithi006.stdout:Nov 17 00:22:14 smithi006 bash[263757]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-a 2023-11-17T00:22:15.183 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.a.service' 2023-11-17T00:22:15.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:15.271 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-17T00:22:15.271 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-17T00:22:15.272 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.d 2023-11-17T00:22:15.473 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:22:15 smithi006 systemd[1]: Stopping Ceph mds.d for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:16.223 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:22:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-d[130895]: 2023-11-17T00:22:15.855+0000 7f5b5a37c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:16.223 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:22:15 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-d[130895]: 2023-11-17T00:22:15.855+0000 7f5b5a37c700 -1 mds.d *** got signal Terminated *** 2023-11-17T00:22:20.473 INFO:journalctl@ceph.mds.d.smithi006.stdout:Nov 17 00:22:20 smithi006 bash[263902]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-d 2023-11-17T00:22:20.606 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.d.service' 2023-11-17T00:22:20.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:20.694 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-17T00:22:20.695 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-17T00:22:20.695 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.g 2023-11-17T00:22:20.973 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:22:20 smithi006 systemd[1]: Stopping Ceph mds.g for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:21.723 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:22:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-g[131506]: 2023-11-17T00:22:21.229+0000 7f82c090b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:21.723 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:22:21 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-g[131506]: 2023-11-17T00:22:21.229+0000 7f82c090b700 -1 mds.g *** got signal Terminated *** 2023-11-17T00:22:23.723 INFO:journalctl@ceph.mds.g.smithi006.stdout:Nov 17 00:22:23 smithi006 bash[264055]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-g 2023-11-17T00:22:23.864 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.g.service' 2023-11-17T00:22:23.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:23.949 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-17T00:22:23.949 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-17T00:22:23.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.j 2023-11-17T00:22:24.223 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:22:24 smithi006 systemd[1]: Stopping Ceph mds.j for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:24.973 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:22:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-j[130509]: 2023-11-17T00:22:24.479+0000 7f01e550d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:24.973 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:22:24 smithi006 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-j[130509]: 2023-11-17T00:22:24.479+0000 7f01e550d700 -1 mds.j *** got signal Terminated *** 2023-11-17T00:22:28.973 INFO:journalctl@ceph.mds.j.smithi006.stdout:Nov 17 00:22:28 smithi006 bash[264199]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-j 2023-11-17T00:22:29.292 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.j.service' 2023-11-17T00:22:29.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:29.377 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-17T00:22:29.377 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-17T00:22:29.377 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.b 2023-11-17T00:22:29.727 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:22:29 smithi087 systemd[1]: Stopping Ceph mds.b for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:30.101 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:22:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:22:29.725+0000 7f0476c87700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:30.101 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:22:29 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b[156468]: 2023-11-17T00:22:29.725+0000 7f0476c87700 -1 mds.b *** got signal Terminated *** 2023-11-17T00:22:32.601 INFO:journalctl@ceph.mds.b.smithi087.stdout:Nov 17 00:22:32 smithi087 bash[164145]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-b 2023-11-17T00:22:32.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.b.service' 2023-11-17T00:22:32.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:32.721 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-17T00:22:32.721 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-17T00:22:32.722 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.e 2023-11-17T00:22:33.094 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:22:32 smithi087 systemd[1]: Stopping Ceph mds.e for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:33.351 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:22:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-e[155788]: 2023-11-17T00:22:33.091+0000 7f9fdcd9c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:33.351 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:22:33 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-e[155788]: 2023-11-17T00:22:33.091+0000 7f9fdcd9c700 -1 mds.e *** got signal Terminated *** 2023-11-17T00:22:34.851 INFO:journalctl@ceph.mds.e.smithi087.stdout:Nov 17 00:22:34 smithi087 bash[164294]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-e 2023-11-17T00:22:34.945 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.e.service' 2023-11-17T00:22:34.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:34.980 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-17T00:22:34.981 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-17T00:22:34.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h 2023-11-17T00:22:35.350 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:22:35 smithi087 systemd[1]: Stopping Ceph mds.h for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:35.601 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:22:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[160432]: 2023-11-17T00:22:35.348+0000 7f8ac7f74700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:35.601 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:22:35 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h[160432]: 2023-11-17T00:22:35.348+0000 7f8ac7f74700 -1 mds.h *** got signal Terminated *** 2023-11-17T00:22:39.848 INFO:journalctl@ceph.mds.h.smithi087.stdout:Nov 17 00:22:39 smithi087 bash[164434]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-h 2023-11-17T00:22:39.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.h.service' 2023-11-17T00:22:39.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:39.888 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-17T00:22:39.888 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-17T00:22:39.888 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.k 2023-11-17T00:22:40.101 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:22:39 smithi087 systemd[1]: Stopping Ceph mds.k for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:40.605 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:22:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-k[155444]: 2023-11-17T00:22:40.291+0000 7fc04909f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:40.613 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:22:40 smithi087 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-k[155444]: 2023-11-17T00:22:40.291+0000 7fc04909f700 -1 mds.k *** got signal Terminated *** 2023-11-17T00:22:42.385 INFO:journalctl@ceph.mds.k.smithi087.stdout:Nov 17 00:22:42 smithi087 bash[164584]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-k 2023-11-17T00:22:42.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.k.service' 2023-11-17T00:22:42.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:42.531 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-17T00:22:42.531 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-17T00:22:42.531 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.c 2023-11-17T00:22:42.866 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:22:42 smithi186 systemd[1]: Stopping Ceph mds.c for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:43.366 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:22:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-c[157148]: 2023-11-17T00:22:42.959+0000 7f5240ea0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:43.366 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:22:42 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-c[157148]: 2023-11-17T00:22:42.959+0000 7f5240ea0700 -1 mds.c *** got signal Terminated *** 2023-11-17T00:22:47.555 INFO:journalctl@ceph.mds.c.smithi186.stdout:Nov 17 00:22:47 smithi186 bash[165243]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-c 2023-11-17T00:22:47.561 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.c.service' 2023-11-17T00:22:47.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:47.598 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-17T00:22:47.598 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-17T00:22:47.598 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.f 2023-11-17T00:22:47.866 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:47 smithi186 systemd[1]: Stopping Ceph mds.f for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:48.366 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-f[156806]: 2023-11-17T00:22:47.974+0000 7ff313388700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:48.366 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:47 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-f[156806]: 2023-11-17T00:22:47.974+0000 7ff313388700 -1 mds.f *** got signal Terminated *** 2023-11-17T00:22:52.552 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.f.service' 2023-11-17T00:22:52.576 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:52 smithi186 bash[165398]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-f 2023-11-17T00:22:52.577 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:52 smithi186 systemd[1]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.f.service: Succeeded. 2023-11-17T00:22:52.577 INFO:journalctl@ceph.mds.f.smithi186.stdout:Nov 17 00:22:52 smithi186 systemd[1]: Stopped Ceph mds.f for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7. 2023-11-17T00:22:52.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:52.589 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-17T00:22:52.589 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-17T00:22:52.589 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.i 2023-11-17T00:22:52.866 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:22:52 smithi186 systemd[1]: Stopping Ceph mds.i for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:53.365 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:22:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: 2023-11-17T00:22:52.965+0000 7f2bfb3e4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:53.366 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:22:52 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i[157494]: 2023-11-17T00:22:52.965+0000 7f2bfb3e4700 -1 mds.i *** got signal Terminated *** 2023-11-17T00:22:57.610 INFO:journalctl@ceph.mds.i.smithi186.stdout:Nov 17 00:22:57 smithi186 bash[165550]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-i 2023-11-17T00:22:57.615 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.i.service' 2023-11-17T00:22:57.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:22:57.649 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-17T00:22:57.649 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-17T00:22:57.650 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.l 2023-11-17T00:22:57.866 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:22:57 smithi186 systemd[1]: Stopping Ceph mds.l for dbc2f8cc-84db-11ee-8dbe-212e2dc638e7... 2023-11-17T00:22:58.366 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:22:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-l[156467]: 2023-11-17T00:22:58.023+0000 7f2303609700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-17T00:22:58.366 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:22:58 smithi186 ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-l[156467]: 2023-11-17T00:22:58.023+0000 7f2303609700 -1 mds.l *** got signal Terminated *** 2023-11-17T00:23:00.473 INFO:journalctl@ceph.mds.l.smithi186.stdout:Nov 17 00:23:00 smithi186 bash[165697]: ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7-mds-l 2023-11-17T00:23:00.478 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dbc2f8cc-84db-11ee-8dbe-212e2dc638e7@mds.l.service' 2023-11-17T00:23:00.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-17T00:23:00.513 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-17T00:23:00.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-17T00:23:05.047 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-17T00:23:06.987 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-17T00:23:08.959 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:23:09.040 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:23:09.077 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-17T00:23:09.107 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-17T00:23:09.109 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi006/crash 2023-11-17T00:23:09.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-17T00:23:09.196 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-17T00:23:09.197 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2023-11-17T00:23:09.200 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi087/crash 2023-11-17T00:23:09.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-17T00:23:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-17T00:23:09.238 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2023-11-17T00:23:09.240 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi186/crash 2023-11-17T00:23:09.241 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-17T00:23:09.276 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-17T00:23:09.276 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2023-11-17T00:23:09.277 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-17T00:23:09.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-11-17T00:23:09.363 INFO:teuthology.orchestra.run.smithi006.stderr:grep: /var/log/ceph/dbc2f8cc-84db-11ee-8dbe-212e2dc638e7/ceph.log: No such file or directory 2023-11-17T00:23:09.368 INFO:tasks.cephadm:Compressing logs... 2023-11-17T00:23:09.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:23:09.373 DEBUG:teuthology.orchestra.run.smithi087:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:23:09.376 DEBUG:teuthology.orchestra.run.smithi186:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:23:09.402 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-17T00:23:09.420 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-17T00:23:09.441 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-17T00:28:46.524 INFO:tasks.cephadm:Archiving logs... 2023-11-17T00:28:46.525 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi006/log 2023-11-17T00:28:46.526 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-17T00:29:09.161 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi087/log 2023-11-17T00:29:09.162 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-17T00:29:52.245 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi186/log 2023-11-17T00:29:52.246 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-17T00:30:16.573 INFO:tasks.cephadm:Removing cluster... 2023-11-17T00:30:16.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force 2023-11-17T00:30:18.366 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force 2023-11-17T00:30:19.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dbc2f8cc-84db-11ee-8dbe-212e2dc638e7 --force 2023-11-17T00:30:20.603 INFO:tasks.cephadm:Removing cephadm ... 2023-11-17T00:30:20.603 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-17T00:30:20.653 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-17T00:30:20.674 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-17T00:30:20.692 INFO:tasks.cephadm:Teardown complete 2023-11-17T00:30:20.693 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-17T00:30:20.708 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-17T00:30:20.708 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-17T00:30:20.713 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-17T00:30:20.717 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-17T00:30:20.875 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-17T00:30:20.886 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-17T00:30:21.113 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-17T00:30:21.118 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-17T00:30:21.121 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-17T00:30:21.137 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-11-17T00:30:21.138 INFO:teuthology.orchestra.run.smithi087.stderr:bash: ntpq: command not found 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +1002us[+1002us] +/- 76ms 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -929us[ -894us] +/- 76ms 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -41us[-5872ns] +/- 87ms 2023-11-17T00:30:21.142 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-17T00:30:21.159 INFO:teuthology.orchestra.run.smithi006.stderr:bash: ntpq: command not found 2023-11-17T00:30:21.165 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-17T00:30:21.165 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2023-11-17T00:30:21.165 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +1000us[+1000us] +/- 76ms 2023-11-17T00:30:21.166 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -968us[ -965us] +/- 76ms 2023-11-17T00:30:21.166 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -82us[ -82us] +/- 87ms 2023-11-17T00:30:21.166 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +999us[+1006us] +/- 76ms 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -941us[ -935us] +/- 76ms 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -44us[ -38us] +/- 87ms 2023-11-17T00:30:21.190 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-17T00:30:21.191 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-17T00:30:21.203 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-17T00:30:21.204 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-17T00:30:21.217 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-17T00:30:21.605 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-17T00:30:21.734 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-17T00:30:21.837 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-17T00:30:21.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:21.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-17T00:30:21.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:21.951 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-17T00:30:21.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:21.985 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-17T00:30:21.998 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-17T00:30:22.015 INFO:teuthology.task.internal:Duration was 2955.497268 seconds 2023-11-17T00:30:22.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-17T00:30:22.026 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-17T00:30:22.027 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-17T00:30:22.034 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-17T00:30:22.036 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-17T00:30:22.083 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-17T00:30:22.113 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-17T00:30:22.185 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-17T00:30:22.398 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-17T00:30:22.398 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2023-11-17T00:30:22.399 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-17T00:30:22.519 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2023-11-17T00:30:22.519 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-17T00:30:22.548 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-11-17T00:30:22.548 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-17T00:30:22.570 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-17T00:30:22.570 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:30:22.629 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:30:22.632 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-17T00:30:22.783 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-17T00:30:22.783 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-17T00:30:22.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-17T00:30:22.808 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-17T00:30:22.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-17T00:30:22.991 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-17T00:30:22.991 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-17T00:30:23.097 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-17T00:30:23.127 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-17T00:30:23.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-17T00:30:23.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.172 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.174 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2023-11-17T00:30:23.260 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2023-11-17T00:30:23.284 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-11-17T00:30:23.328 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:23.370 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:23.513 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-17T00:30:23.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-17T00:30:23.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-17T00:30:23.545 INFO:teuthology.task.internal:Transferring archived files... 2023-11-17T00:30:23.546 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi006 2023-11-17T00:30:23.546 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-17T00:30:23.636 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi087 2023-11-17T00:30:23.636 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-17T00:30:23.678 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180/remote/smithi186 2023-11-17T00:30:23.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-17T00:30:23.718 INFO:teuthology.task.internal:Removing archive directory... 2023-11-17T00:30:23.719 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-17T00:30:23.723 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-17T00:30:23.872 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-17T00:30:23.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-17T00:30:23.903 INFO:teuthology.task.internal:Not uploading archives. 2023-11-17T00:30:23.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-17T00:30:23.916 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-17T00:30:23.916 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-17T00:30:23.921 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-17T00:30:23.931 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-17T00:30:23.947 INFO:teuthology.orchestra.run.smithi087.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 17 00:30 /home/ubuntu/cephtest 2023-11-17T00:30:23.947 INFO:teuthology.orchestra.run.smithi186.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 17 00:30 /home/ubuntu/cephtest 2023-11-17T00:30:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 17 00:30 /home/ubuntu/cephtest 2023-11-17T00:30:23.966 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-17T00:30:23.979 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-17T00:30:24.008 INFO:teuthology.nuke:Checking targets against current locks 2023-11-17T00:30:24.035 DEBUG:teuthology.nuke:shortname: smithi006 2023-11-17T00:30:24.035 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-17T00:30:24.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.707568', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-17T00:30:27.585 DEBUG:teuthology.nuke:shortname: smithi087 2023-11-17T00:30:27.692 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-17T00:30:27.715 DEBUG:teuthology.nuke:shortname: smithi186 2023-11-17T00:30:27.716 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-17T00:30:27.720 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.711857', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-17T00:30:27.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_21:22:00-fs-wip-choffman-62793-distro-default-smithi/7461180', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 23:32:08.710219', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-17T00:30:27.751 INFO:teuthology.orchestra.console.smithi006:Power off 2023-11-17T00:30:27.752 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-17T00:30:27.787 INFO:teuthology.orchestra.console.smithi087:Power off 2023-11-17T00:30:27.788 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-17T00:30:27.801 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-11-17T00:30:27.801 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:27.812 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-11-17T00:30:27.812 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:27.824 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-17T00:30:28.446 INFO:teuthology.orchestra.console.smithi186:Power off 2023-11-17T00:30:28.572 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-17T00:30:28.650 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-17T00:30:28.688 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-11-17T00:30:28.702 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:29.134 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-17T00:30:32.439 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:32.550 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-17T00:30:32.687 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:32.797 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-17T00:30:33.136 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:33.248 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-17T00:30:36.554 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:36.667 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-11-17T00:30:36.768 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-11-17T00:30:36.798 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:36.910 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-11-17T00:30:37.010 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-11-17T00:30:37.250 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-17T00:30:37.360 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-17T00:30:37.461 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-11-17T00:30:37.567 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} duration: 2955.4972684383392 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi006 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1c9565146c8647d4e00eb6c7ae626dc192680e7b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c4ef3c3277f54ce6ad59166747e24568 status: fail success: false 2023-11-17T00:30:37.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-17T00:30:37.649 INFO:teuthology.run:FAIL